System, method, and device of authenticating a user based on selfie image or selfie video

ABSTRACT

System, method, and device of detecting identity of a user and authenticating a user; as well as detecting a possible attacker or impostor, and differentiating among users of an electronic device or of a computerized service. A mobile or portable electronic device is utilized to capture a self-taken image or video of a user, which is utilized as a user-authentication factor. The accelerometer and gyroscope or device-orientation sensor of the mobile device, sense and measure spatial and physical device properties during, before or after the submission of the self-taken image or video. Based on such spatial and physical device properties, in combination with computer-vision analysis of the content shown in the self-taken image or video, the system determines liveness of the user and freshness of the submitted self-taken image or video, and differentiates between a legitimate user and an attacker.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a Continuation-in-Part (CIP) of U.S. Ser. No.15/294,951, filed on Oct. 17, 2016, which is hereby incorporated byreference in its entirety.

The above-mentioned U.S. Ser. No. 15/294,951 is a Continuation-in-Part(CIP) of U.S. Ser. No. 14/727,873, filed on Jun. 2, 2015, now U.S. Pat.No. 9,526,006; which was a Continuation-in-Part (CIP) of U.S. Ser. No.14/566,723, filed on Dec. 11, 2014, now U.S. Pat. No. 9,071,969; whichis a Continuation of U.S. Ser. No. 13/922,271, filed on Jun. 20, 2013,now U.S. Pat. No. 8,938,787; which is a Continuation-In-Part (CIP) ofU.S. Ser. No. 13/877,676, filed on Apr. 4, 2013, now U.S. Pat. No.9,069,942; which is a National Stage filing of PCT InternationalApplication number PCT/IL2011/000907, filed on Nov. 29, 2011; whichclaims priority and benefit from U.S. 61/417,479, filed on Nov. 29,2010; and all of the above-mentioned patent applications are herebyincorporated by reference in their entirety.

The above-mentioned U.S. Ser. No. 15/294,951 also claims priority andbenefit from U.S. 62/308,878, filed on Mar. 16, 2016, which is herebyincorporated by reference in its entirety.

This patent application is a Continuation-in-Part (CIP) of U.S. Ser. No.16/416,222, filed on May 19, 2019, which is hereby incorporated byreference in its entirety.

The above-mentioned U.S. Ser. No. 16/416,222 is a Continuation-in-Part(CIP) of U.S. Ser. No. 15/708,155, filed on Sep. 19, 2017, now U.S. Pat.No. 10,298,614, which is hereby incorporated by reference in itsentirety.

The above-mentioned U.S. Ser. No. 15/708,155 is a Continuation-In-Part(CIP) of U.S. Ser. No. 15/422,479, filed on Feb. 2, 2017, now U.S. Pat.No. 9,779,423, which is hereby incorporated by reference in itsentirety.

The above-mentioned U.S. Ser. No. 15/422,479 claims priority and benefitfrom U.S. 62/312,140, filed on Mar. 23, 2016, which is herebyincorporated by reference in its entirety.

The above-mentioned U.S. Ser. No. 15/422,479 is also aContinuation-in-Part (CIP) of U.S. Ser. No. 15/276,803, filed Sep. 27,2016, now U.S. Pat. No. 10,055,560, which is hereby incorporated byreference in its entirety.

The above-mentioned U.S. Ser. No. 15/276,803 is a Continuation-in-Part(CIP) of U.S. Ser. No. 14/325,398, filed on Jul. 8, 2014, now U.S. Pat.No. 9,477,826, which is hereby incorporated by reference in itsentirety.

The above-mentioned U.S. Ser. No. 14/325,398 claims priority and benefitfrom U.S. 61/843,915, filed on Jul. 9, 2013, which is herebyincorporated by reference in its entirety.

The above-mentioned U.S. Ser. No. 14/325,398 is a Continuation-in-Part(CIP) of U.S. Ser. No. 13/922,271, filed on Jun. 20, 2013, now U.S. Pat.No. 8,938,787, which is hereby incorporated by reference in itsentirety.

The above-mentioned U.S. Ser. No. 14/325,398 is a Continuation-in-Part(CIP) of U.S. Ser. No. 13/877,676, filed on Apr. 4, 2013, now U.S. Pat.No. 9,069,942; which was a National Phase of PCT InternationalApplication number PCT/IL2011/000907, filed on Nov. 29, 2011; whichclaimed priority and benefit from U.S. 61/417,479, filed on Nov. 29,2010; all of which are hereby incorporated by reference in theirentirety.

The above-mentioned U.S. Ser. No. 14/325,398 is a Continuation-in-Part(CIP) of U.S. Ser. No. 14/320,653, filed on Jul. 1, 2014, now U.S. Pat.No. 9,275,337, which is hereby incorporated by reference in itsentirety.

The above-mentioned U.S. Ser. No. 14/325,398 is a Continuation-in-Part(CIP) of U.S. Ser. No. 14/320,656, filed on Jul. 1, 2014, now U.S. Pat.No. 9,665,703, which is hereby incorporated by reference in itsentirety.

The above-mentioned U.S. Ser. No. 15/422,479 is also aContinuation-in-Part (CIP) of U.S. Ser. No. 15/210,221, filed Jul. 14,2016, now U.S. Pat. No. 9,674,218, which is hereby incorporated byreference in its entirety.

The above-mentioned U.S. Ser. No. 15/210,221 is a Continuation of U.S.Ser. No. 14/675,768, filed on Apr. 1, 2015, now U.S. Pat. No. 9,418,221,which is hereby incorporated by reference in its entirety.

The above-mentioned U.S. Ser. No. 14/675,768 claims priority and benefitfrom U.S. 61/973,855, filed on Apr. 2, 2014, which is herebyincorporated by reference in its entirety.

The above-mentioned U.S. Ser. No. 14/675,768 is a Continuation-in-Part(CIP) of U.S. Ser. No. 14/566,723, filed on Dec. 11, 2014, now U.S. Pat.No. 9,071,969; which is a Continuation of U.S. Ser. No. 13/922,271,filed on Jun. 20, 2013, now U.S. Pat. No. 8,938,787; which is aContinuation-in-Part (CIP) of U.S. Ser. No. 13/877,676, filed on Apr. 4,2013, now U.S. Pat. No. 9,069,942; which is a National Stage of PCTInternational Application number PCT/IL2011/000907, having anInternational Filing Date of Nov. 29, 2011; which claims priority andbenefit from U.S. 61/417,479, filed on Nov. 29, 2010; all of which arehereby incorporated by reference in their entirety.

The above-mentioned U.S. Ser. No. 16/416,222 is also aContinuation-in-Part (CIP) of U.S. Ser. No. 15/368,608, filed on Dec. 4,2016, which is hereby incorporated by reference in its entirety.

The above-mentioned U.S. Ser. No. 15/368,608 is a Continuation-in-Part(CIP) of U.S. Ser. No. 15/001,259, filed on Jan. 20, 2016, now U.S. Pat.No. 9,541,995; which is a Continuation of U.S. Ser. No. 14/320,653,filed on Jul. 1, 2014, now U.S. Pat. No. 9,275,337; all of which arehereby incorporated by reference in their entirety.

The above-mentioned U.S. Ser. No. 14/320,653 claims priority and benefitfrom U.S. 61/843,915, filed on Jul. 9, 2013, which is herebyincorporated by reference in its entirety.

The above-mentioned U.S. Ser. No. 14/320,653 is also aContinuation-in-Part (CIP) of U.S. Ser. No. 13/922,271, filed on Jun.20, 2013, now U.S. Pat. No. 8,938,787, which is hereby incorporated byreference in its entirety.

The above-mentioned U.S. Ser. No. 14/320,653 is also aContinuation-in-Part (CIP) of U.S. Ser. No. 13/877,676, filed on Apr. 4,2013, now U.S. Pat. No. 9,069,942, which was a National Phase of PCTInternational Application number PCT/IL2011/000907, filed on Nov. 29,2011, which claimed priority and benefit from U.S. 61/417,479, filed onNov. 29, 2010. All of the above-mentioned patent applications are herebyincorporated by reference in their entirety.

The above-mentioned U.S. Ser. No. 15/368,608 is also aContinuation-in-Part (CIP) of U.S. patent application Ser. No.14/727,873, filed on Jun. 2, 2015, now U.S. Pat. No. 9,526,006, which ishereby incorporated by reference in its entirety.

The above-mentioned U.S. Ser. No. 15/368,608 is also aContinuation-in-Part (CIP) of U.S. Ser. No. 15/360,291, filed on Nov.23, 2016, now U.S. Pat. No. 9,747,436; which is a Continuation-in-Part(CIP) of U.S. Ser. No. 14/718,096, filed on May 21, 2015, now U.S. Pat.No. 9,531,701; all of which are hereby incorporated by reference intheir entirety.

This patent application is also a Continuation-in-Part (CIP) of U.S.Ser. No. 16/242,015, filed on Jan. 8, 2019, which is hereby incorporatedby reference in its entirety.

The above-mentioned U.S. Ser. No. 16/242,015 claims benefit and priorityfrom U.S. 62/621,600, filed on Jan. 25, 2018, which is herebyincorporated by reference in its entirety.

The above-mentioned U.S. Ser. No. 16/242,015 is a Continuation-in-Part(CIP) of U.S. Ser. No. 16/057,825, filed on Aug. 8, 2018, which ishereby incorporated by reference in its entirety. This patentapplication claims priority and/or benefit from all the patentapplications from which U.S. Ser. No. 16/057,825 had directly orindirectly claimed priority and/or benefit.

The above-mentioned U.S. Ser. No. 16/242,015 is also aContinuation-in-Part (CIP) of U.S. Ser. No. 15/885,819, filed on Feb. 1,2018, which is hereby incorporated by reference in its entirety. Thispatent application claims priority and/or benefit from all the patentapplications from which U.S. Ser. No. 15/885,819 had directly orindirectly claimed priority and/or benefit.

The above-mentioned U.S. Ser. No. 16/242,015 is also aContinuation-in-Part (CIP) of U.S. Ser. No. 15/368,608, filed on Dec. 4,2016, which is hereby incorporated by reference in its entirety. Thispatent application claims priority and/or benefit from all the patentapplications from which U.S. Ser. No. 15/368,608 had directly orindirectly claimed priority and/or benefit.

FIELD

The present invention is related to the security of electronic devicesand systems.

BACKGROUND

Millions of people around the world utilize mobile electronic devices,such as smartphones and tablets, in order to perform various activities.Such activities may include, for example, browsing the Internet, sendingand receiving electronic mail (email) messages, taking photographs andvideos, engaging in a video conference or a chat session, playing games,or the like.

Some activities may be privileged, or may require authentication of theuser in order to ensure that only an authorized user engages in theactivity. For example, a user may be required to enter a username and apassword in order to access an email account, or in order to access anonline banking interface or website.

SUMMARY

Some embodiments of the invention comprise a system, a method, and adevice of detecting identity of a user and authenticating a user; aswell as detecting a possible attacker or impostor, and differentiatingamong users of an electronic device or of a computerized service.

For example, a mobile or portable electronic device is utilized tocapture a self-taken image of a user, which is utilized as auser-authentication factor. The accelerometer and gyroscope of themobile device, sense and measure geo-spatial device properties during,before or after the submission of the self-image authentication factor;and based on such geo-spatial device properties, the system determinesliveness of the user, freshness of the submitted self-image, andpossibly differentiates between a legitimate user and an attacker.

Similarly, geo-spatial device properties that accompany the entry or thesubmission of a biometric sample, such as fingerprint scan or retinascan, are used to detect liveness or freshness or authenticity of suchbiometric samples, and are used to differentiate between a legitimateuser and an attacker.

The present invention may provide other and/or additional benefits oradvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram illustration of a mobile electronicdevice, in accordance with some demonstrative embodiments of the presentinvention.

FIG. 2 is an illustration of three graphs, which demonstrateacceleration as a function of time over three separate axes, inaccordance with some demonstrative embodiments of the present invention.

FIG. 3 is an illustration of a graph of the main axes ofdimension-reduced space of accelerometer reaction to tapping, inaccordance with some demonstrative embodiments of the present invention.

FIG. 4 is an illustration of a graph depicting feature space, inaccordance with some demonstrative embodiments of the present invention.

FIG. 5 is a flow-chart of a method, in accordance with somedemonstrative embodiments of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention may include, for example, systems, devices, andmethods for detecting identity of a user of a mobile electronic device,and for determining that a mobile electronic device is used by afraudulent user.

In accordance with the present invention, for example, a method forconfirming identity of a user of a mobile electronic device maycomprise: receiving touch data from a touch-screen of the mobileelectronic device; receiving acceleration data from an accelerometer ofthe mobile electronic device; correlating between the touch data and theacceleration data; based on the correlating, generating a user-specifictrait indicative of said user.

In accordance with the present invention, for example, the method maycomprise: storing a reference value of the user-specific trait,indicative of said user; in a subsequent usage session of the mobileelectronic device, generating a current value of the user-specific traitcorrelating between touch data and acceleration data; and based on acomparison between the current value of the user-specific trait and thereference value of the user-specific trait, determining whether or not acurrent user of the mobile electronic device is an authorized user ofthe mobile electronic device.

In accordance with the present invention, for example, storingcomprises: storing within said mobile electronic device; and saidcomparison is performed within said mobile electronic device.

In accordance with the present invention, for example, storing comprisesstoring externally to said mobile electronic device; and said comparisonis performed externally to said mobile electronic device, and compriseswirelessly receiving at the mobile electronic device an indication ofsaid comparison.

In accordance with the present invention, for example, said touch datacomprises non-tactile touch data indicating a hovering user gesture inproximity to said touch-screen.

In accordance with the present invention, for example, the method maycomprise: receiving gyroscope data from a gyroscope of the mobileelectronic device; correlating between the touch data and the gyroscopedata; based on the correlating between the touch data and the gyroscopedata, generating another user-specific trait indicative of said user.

In accordance with the present invention, for example, the method maycomprise: capturing non-tactile motion data indicating a user gesture;correlating between the non-tactile motion data and the accelerationdata; based on the correlating between the non-tactile motion data andthe acceleration data, generating another user-specific trait indicativeof said user.

In accordance with the present invention, for example, the method maycomprise: comparing between (a) a currently-calculated value of theuser-specific trait, corresponding to a current usage of the mobileelectronic device, and (b) a previously-calculated value of theuser-specific trait, corresponding to a previous usage of the mobileelectronic device; and based on a comparison result, performing at leastone of: restricting access of said user to an online service;restricting access of said user to an application installed on saidmobile electronic device; requiring the user to authenticate hisidentity to an online service; requiring the user to authenticate hisidentity to an application installed on said mobile electronic device.

In accordance with the present invention, for example, the method maycomprise: based on said touch data, estimating user-specific motorcontrol parameters and user-specific motor control noise; and based onthe estimated user-specific motor control parameters and user-specificmotor control noise, differentiating between said user and another user.

In accordance with the present invention, for example, the method maycomprise: based on said touch data, estimating user-specific motorcontrol parameters and user-specific motor control noise of a controlloop which comprises translation error and gesture velocity error; andbased on the estimated user-specific motor control parameters anduser-specific motor control noise, differentiating between said user andanother user.

In accordance with the present invention, for example, the method maycomprise: based on said correlating, estimating a user-specificphysiological trait of said user; and based on the user-specificphysiological trait, differentiating between said user and another user.

In accordance with the present invention, for example, estimating theuser-specific physiological trait of said user comprises at least oneof: estimating a length of a finger of the user; estimating a width of afinger of the user; estimating a size-related parameter of a finger ofthe user; estimating a distance between a tip of a finger of the userand another part of a hand of the user.

In accordance with the present invention, for example, the method maycomprise: based on said correlating, estimating a user-specificbehavioral trait of said user; and based on the user-specific behavioraltrait, differentiating between said user and another user.

In accordance with the present invention, for example, estimating theuser-specific behavioral trait of said user comprises: determining thatsaid user typically performs a particular inadvertent gesture whileperforming a user-intended input-providing gesture.

In accordance with the present invention, for example, estimating theuser-specific behavioral trait of said user comprises one or more of:determining that said user typically moves the mobile electronic deviceat a particular direction while performing a touch gesture; determiningthat said user typically rotates the mobile electronic device whileperforming a touch gesture; determining that said user typically slantsthe mobile electronic device at a particular angle while performing atouch gesture.

In accordance with the present invention, for example, estimating theuser-specific behavioral trait of said user comprises: determining thatsaid user typically holds the mobile electronic device with a first handof the user and concurrently performs an input-providing gesture with asecond hand of the user.

In accordance with the present invention, for example, estimating theuser-specific behavioral trait of said user comprises: determining thatsaid user typically holds the mobile electronic device with a singlehand and concurrently performs an input-providing gesture with saidsingle hand.

In accordance with the present invention, for example, the method maycomprise: based on said correlating, estimating a first user-specificbehavioral trait of said user which corresponds to a first usagescenario; based on said correlating, estimating a second user-specificbehavioral trait of said user which corresponds to a second usagescenario; based on the first and second user-specific behavioral traits,differentiating between said user and another user.

In accordance with the present invention, for example, the method maycomprise: based on said correlating, estimating a first user-specificbehavioral trait of said user which corresponds to a first usagescenario in which said user operates said mobile electronic device whilethe user holds said mobile electronic device; based on said correlating,estimating a second user-specific behavioral trait of said user whichcorresponds to a second usage scenario in which said user operates saidmobile electronic device while the user does not hold said mobileelectronic device; based on the first and second user-specificbehavioral traits, differentiating between said user and another user.

In accordance with the present invention, for example, a mobileelectronic device may be configured to confirm identity of a user ofsaid mobile electronic device; the mobile electronic device comprising:a touch-screen to receive touch data; an accelerometer to receiveacceleration data; a correlator module to correlate between the touchdata and the acceleration data; a trait extractor module to generate auser-specific trait indicative of said user, based on correlationbetween the touch data and the acceleration data.

Applicants have realized that each user of a mobile electronic devicemay handle the device in a unique manner which may be detected and maybe utilized for confirming the identity of the user, or for othersecurity-related purposes or fraud-detection purposes. Applicants haverealized, for example, that different users cause different type ofacceleration to the mobile device when they perform the same operationor touch-gesture (e.g., swiping or tapping or scrolling on thetouch-screen), or may tilt or rotate or slant the mobile device indifferent, unique ways when they perform such gestures or operations.

The present invention may include, for example, biometric modalities,personal trait extraction modalities, and/or identity authenticationmodalities which may be used in conjunction with a mobile or portableelectronic device, and may utilize a combination of (or correlationbetween) acceleration parameters and/or touch data. Such parameters maybe used in order to deduce unique insights regarding the identity orpossible identity of the user of the mobile electronic device, or inorder to determine whether or not the user is considered to be the“genuine” user, or in contrast, an attacker or impersonator or“fraudster”.

The present invention may capture, monitor, or otherwise utilize fordeduction of insights, the coupling or correlation between (a)touch-screen interaction, or other user gestures, and (b)accelerometer(s) measurements and/or gyroscope(s) measurements. Thepresent invention may further deduce and/or utilize one or more otherbiometric traits or identity-authentication traits, for example, touchor swipe locations, pressure dynamics, identification of physiologicalregions (e.g., in the hand of the user) that move while other regions donot move when a user gesture is performed, or other suitable traits inorder to assist in identification and/or authentication of the user ofthe mobile device.

The present invention may sufficiently capture unique qualities of ahuman user to be usable as a biometric for authentication. Differentpeople may have different preferred orientations for holding or grasping(e.g., in their hand) a mobile device, and/or a different way in whichthey press or touch or tap the touch-screen (e.g., the applied force,the duration of the tapping, or the like).

Applicants have realized that physical traits such as, for example, handsize, hand mass, or other traits, may change the way in which a user'sinteracting hand and his device-holding hand are correlated. In ademonstrative example, the present invention may distinguish ordifferentiate between (a) a person who is using one single hand for bothholding the mobile device and tapping on its touch-screen (or performingother touch gesture), and (b) a person who is using one hand to hold themobile device and another hand to tap on its touch-screen (or to performother touch gesture or user gesture).

Moreover, as Applicants have realized, different tap locations (e.g.,top-left corner or region of the touch-screen, versus bottom-rightcorner or region) may create different torque(s) on the mobile device,further depending on the tap strength, the offset of the mobile devicein the hand (e.g., the device being held high or low, with the palm areaor the fingers area, or the like) and/or the size of the hand (e.g., ifthe same hand is used for both holding the device and tapping on itstouch-screen).

The terms “mobile device” or “mobile electronic device” as used hereinmay include, for example, a smartphone, a cellular phone, a mobilephone, a tablet, a handheld device, a portable electronic device, aportable gaming device, a portable audio/video player, a smart-watch, adigital watch, a digital wrist-watch, an Augmented Reality (AR) orVirtual Reality (VR) device or glasses or helmet or headset (e.g.,similar to Google Glass, or similar to Oculus Rift), a fitness band orfitness watch, a laptop computer, a tablet computer, a notebookcomputer, a netbook computer, an electronic device which comprises atleast an accelerometer and a touch-screen, or the like.

The term “genuine user” as used herein may include, for example, anowner of a mobile electronic device; a legal or lawful user of a mobileelectronic device; an authorized user of a mobile electronic device; aperson who has legal authorization and/or legal right to utilize amobile electronic device, for general purpose(s) and/or for one or moreparticular purpose(s); or the person who had originally defined usercredentials (e.g., username and password) for performing an activitythrough the mobile electronic device.

The term “fraudulent user” as used herein may include, for example, anyperson who is not the “genuine user” of the mobile electronic device; anattacker; an intruder; a man-in-the-middle attacker; aman-in-the-browser attacker; an unauthorized user; an impersonator; ahacker; a cracker; a person attempting to hack or crack or compromise asecurity measure utilized by the mobile electronic device or utilized byan activity or service accessible through the mobile electronic device;a fraudster; a human fraudster; a “bot” or a malware or an automatedcomputerized process (e.g., implemented by using software modules and/orhardware components) which attempts to imitate human behavior or whichattempts to act as if such “bot” or malware or process was the genuineuser; or the like.

The term “user gesture” as used herein may include, for example, agesture or movement or other operation that a user of a mobile deviceperforms on a touch-screen of the mobile device, or performs inproximity to the touch-screen of the mobile device; touch gesture; tapgesture or double-tap gesture or prolonged tap gesture; scroll gesture;drag gesture, or drag-and-drop gesture; release gesture; click ordouble-click gesture; hovering gestures, in which the user may hoverwith his finger(s) or hand(s) in proximity to the touch-screen of themobile device but without necessarily touching the touch-screen device;hovering gestures that may be captured by a camera of the mobile device,or by a touch-screen of the mobile device (e.g., by taking into accountelectrical and/or magnetic effects of such gestures); hovering gestureswhich may be generally similar to touch-free hovering gestures that aSamsung Galaxy S4 smartphone is able to detect; finger(s) gesturesand/or hand(s) gestures made in a three-dimensional space, for example,similar to movement gestures that a Microsoft Kinect motion sensinginput device is able to sense; and/or a combination of such gestures orother gestures.

Reference is made to FIG. 1, which is a schematic block diagramillustration of a mobile device 100 in accordance with the presentinvention. Mobile device 100 may comprise, for example, a processor 101,a memory unit 102, a storage unit 103, a wireless transceiver 104, atouch-screen 105, one or more accelerometers 106, and one or moregyroscopes 107. Mobile device 100 may further comprise, for example, oneor more hovering sensors 108, one or more motion gesture sensor(s) 109,a correlator 131, a trait extractor 132, a trait repository 133, aprofile constructor module 134, an identity authenticator module 135,and a physiological trait estimator 139. Mobile device 100 may compriseother suitable hardware components and/or software modules, for example,a power source (e.g., a rechargeable battery), an Operating System,software applications, or the like.

Touch-screen 105 may receive user gestures, for example, tapping,double-tapping, dragging, pressing, holding down, releasing, scrolling,pinching fingers for zoom-out, spreading fingers for zoom-in, or thelike). Touch data may be stored in a touch data repository 125,optionally in association with a time-stamp associated with each touchdata-item being stored.

Accelerometer(s) 106 may comprise, for example, a three-axisaccelerometer able to measure acceleration, separately, along three axes(X axis, Y axis, Z axis). Accelerometer readings may be stored in anacceleration data repository 126, optionally in association with atime-stamp associated with each acceleration data-item being stored.

Gyroscope(s) 107 may comprise, for example, a three-axis gyroscope ableto measure orientation and/or rotation, e.g., separately along threeaxes (X axis, Y axis, Z axis). The measured data may be stored in agyroscope data repository 127, optionally in association with atime-stamp associated with each orientation/rotation data-item beingstored.

Hovering sensor(s) 108 may comprise, for example, one or more sensors(e.g., optical sensors, magnetic sensors, electric sensors, touch-screencomponents, camera components, or the like) able to sense hoveringgesture(s) of the user of device 100, for example, in athree-dimensional space or separately along three axes (X axis, Y axis,Z axis). The measured data may be stored in a hovering data repository128, optionally in association with a time-stamp associated with eachhovering data-item being stored.

Motion gesture sensor(s) 109 may comprise, for example, one or moresensors able to sense motion gesture(s) of the user of device 100, forexample, in a three-dimensional space or separately along three axes (Xaxis, Y axis, Z axis). The measured data may be stored in a motiongesture data repository 129, optionally in association with a time-stampassociated with each motion gesture data-item being stored.

Correlator 131 may search for, or identify or determine, correlationamong (a) acceleration data and/or gyroscope data, and (b) touch dataand/or hovering data and/or motion gesture data. Trait extractor 132 maydetermine one or more user-specific traits or characteristics which maybe, or may appear to be, unique to (or indicative of) a particular user,based on one or more correlation(s) identified by correlator 131. Traitvalues or trait indicators, or data indicative of extracteduser-specific traits, may be stored in a trait repository 133.

Profile constructor module 134 may utilize a learning algorithm toconstruct a user profile based on the one or more user-specific traitsidentified by trait extractor 132 and stored in trait repository 133.Profile construction may be performed over a per-defined time period(e.g., five hours, or three days) of the user interacting with device100; or over a pre-defined number of interactions (e.g., 12 or 25 or 100interactions) of the user with device 100. Optionally, profileconstructor module 134 may dynamically extend or shorten or modify therequired time-period or interaction number, for example, if traits of aparticular user are distinctive and are rapidly extracted over a shorterperiod of time or over a smaller number of user interactions.Constructed user profiles may be stored in a user profile repository,which may be internal to device 100 or may be external thereto (e.g., ina remote server or in a “cloud computing” server), optionally with anassociated flag or parameter indicating whether a particular userprofile is fully constructed or under construction.

Identity authenticator module 135 may capture one or more traits of auser who is currently utilizing device 100, and may analyze anddetermine whether or not these traits are similar to, or different from,user-specific traits in a user profile associated with a user that isbelieved to be a “genuine” user of device 100. The analysis results maybe notified by identity authenticator module 135 to other units ormodules, within device 100 (e.g., an application or process running indevice 100) and/or externally to device 100 (e.g., on a remote server,on a remote web-site or web-page, in a “cloud” server or device).

For example, if the analysis indicates that the current user of device100 is not the genuine user, then, one or more fraud-stopping operationsor additional authentication operations may be triggered and performed,for example, requiring the user to re-enter his password or pass-phraseor Personal Identification Number (PIN), requiring the user to answerone or more security questions, requiring the user to perform log-inoperations or to provide account details (e.g., to provide date-of-birthdata), requiring the user to place a phone call to a fraud department ora security department of a service or entity associated with anapplication running on device 100; blocking or restricting or curtailingaccess of the user to one or more services or features which may begenerally available through device 100; or the like.

Correlator 131 may identify user-specific physiological correlations.For example, correlator 131 may identify one or more geometric place(s),on touch-screen 105 or in a space proximate to touch-screen 105, inwhich a user gesture is associated with movement of a user body part(e.g., the thumb; one or more fingers; the palm or wrist) while alsobeing associated with rest or non-movement of other body parts of theuser. Based on the user-specific physiological correlations, traitextractor 132 may extract user-specific physiological trait(s).

In a demonstrative example, trait extractor 132 may determine that forthe user Adam, a vertical scroll-down touch-gesture is typicallyassociated with movement of the root of the thumb, while the otherfingers are at rest and while the wrist or palm-base are at rest;whereas, for the user Bob, a vertical scroll-down touch-gesture istypically associated with both movement of the root of the thumb, aswell as with slight rotational movement of fingers that hold or supportthe rear of the mobile device, and while the wrist or palm-base are atrest. This may be subsequently used for user authentication or foridentity confirmation, to distinguish between a “genuine” user (e.g.,Adam) and a fraudulent user or non-genuine user (e.g., Bob) when theuser of device 100 performs a similar user gesture.

In another demonstrative embodiment, correlator 131 may determine thatthe user of device 100 (e.g., the “genuine” user), while performing aprimary gesture or an intended gesture (e.g., required in order toprovide user input to device 100), typically also performs a secondarygesture an inadvertent gesture (e.g., not required in order to provideuser input to device 100). For example, the primary gesture may be ascrolling gesture, a zoom-in or zoom-out gesture, a dragging gesture, atapping gesture, or other user input gesture; whereas, the secondarygesture (e.g., the inadvertent or unintended gesture, to which the usermay not even be aware) may be, for example, slight or significantrotating or spinning of device 100, slight or significant movement ofdevice 100 (e.g., in a particular direction), slight or significanttilting or slanting of device 100 (e.g., at a particular angle orrange-of-angles), or the like.

In another demonstrative embodiment, correlator 131 may be associatedwith, or may operate in conjunction with, physiological trait estimator139 which may be able to indirectly estimate one or more physiologicaltraits or physiological characteristics of the user of device 100, andparticularly, of the hand(s) or finger(s) (e.g., a finger, a thumb, orthe like) of that user. For example, physiological trait estimator 139may estimate a width of a finger or thumb based on a width of a swipingtrace performed by the finger on touch-screen 105; may estimate a lengthof a finger or thumb based on a radius of a circular or arched or curvedswiping motion on touch-screen 105; may estimate the distance betweenthe tip of a finger or thumb and the palm of the hand, or the wrist; mayestimate other dimensions of hand-parts, or relations between such handparts; or the like. Physiological trait estimator 139 may thus estimatephysiological characteristics which may be unique to a particular user,and may assist in confirming user identity and/or in detecting anon-genuine user impersonating the genuine user.

Additionally or alternatively, correlator 131 may be associated with, ormay operate in conjunction with, a motor control estimator 138 which mayestimate user-specific motor control parameters based on the user'sinteraction with mobile device 100. Such parameters may include, forexample, parameters of the action-perception loop modeling the hand-eyecoordination, as well as control loop parameter, motor noise, perceptionnoise, or the like. Motor control estimator 138 may estimateuser-specific parameters of motor control, which may be more inherent tothe user and may be less action-dependent.

In a demonstrative implementation, for example, motor control estimator138 may track a user gesture on the touch-screen (e.g., a scroll orswipe gesture). The movement or gesture may begin at rest in astart-point (X₀, Y₀), and may end at rest in an end-point (X₁, Y₁). Ademonstrative control loop of the second order, for example, may assumethat the force of the hand is governed by a linear combination of twoerror terms: a translation error, and the current velocity error.

The translation error may be represented as:Δx=(x ₁ −x(t))

The current velocity error may be represented as:

${\Delta\; v_{x}} = {\frac{d}{dt}{x(t)}}$

The control loop may be represented (for the X-axis, and similarly andseparately also for the Y-axis) as:

$\frac{d^{2}{x(t)}}{{dt}^{\; 2}} = {{\alpha_{x}\Delta\; x} + {\beta_{x}v_{x}} + n_{x}}$

In the last equation, α_(x) and β_(x) are control loop parameters, andn_(x) is motor control noise (e.g., Gaussian random variable).Accordingly, motor control estimator 138 may estimate or may simulatetrajectories which may be similar to human trajectories; and although avelocity curve may be different for each movement of the same movement,the velocity curve may be generated by the same model parameters of thatspecific user. Motor control estimator 138 may thus estimate these threeparameters (for the X-axis, and/or for the Y-axis), thereby estimatinguser-specific motor control traits which may be used for differentiatingbetween a genuine user and an impersonator or attacker, regardless ofthe specific movement(s) or gesture(s) performed. The above is only ademonstrative example, and motor control estimator 138 may utilize othermotor control estimations, forward model(s), feedback model(s),estimation of similar peak velocity (or other movement properties) fordifferent movements (e.g., if the error terms are distorted by anon-linear function).

Additionally or alternatively, correlator 131 may identify user-specificbehavioral correlations. For example, correlator 131 may identify thatwhen a particular user performs a particular user-gesture, performanceof the gesture affects in a particular way the acceleration data and/orthe orientation/rotation data of device 100. Based on the user-specificbehavioral correlations, trait extractor 132 may extract user-specificbehavioral trait(s).

In a demonstrative example, trait extractor 132 may determine that forthe user Adam, a horizontal swipe gesture is typically associated with acounter-clockwise rotation in the range of 10 to 15 degrees around avertical axis (e.g., a rotation axis parallel to the longest dimensionof device 100); whereas, for the user Bob, a horizontal swipe gesture istypically associated with a clockwise rotation in the range of 5 to 10degrees (or, with substantially no rotation at all) around that verticalaxis. This may be subsequently used for user authentication or foridentity confirmation, to distinguish between a “genuine” user (e.g.,Adam) and a fraudulent user or non-genuine user (e.g., Bob) when theuser of device 100 performs a similar user gesture.

Correlator 131 may be configured to search for, and detect, otheruser-specific behavioral correlations, for example: correlations basedon the manner of holding device 100 (e.g., a primary angle of holding),and the effect of various user gestures on such holding or on theprimary angle of holding; correlations based on the stability or theshakiness of device 100 (e.g., optionally taking into account the amountand/or frequency and/or timing of hand vibrations or hand movements),and the effect of various user gestures on such device stability orshakiness, or on stability or shakiness of the hand of the user thatholds or operates device 100; correlations based on movement, spinning,rotation and/or acceleration of device 100, along one axis or two axesor three axes, as a result of (or concurrently with) a user gesture suchas, for example, tap, double-tap, prolonged tap, release, drag, drag anddrop, click, double-click, rotation or movement of an on-screen object,rotation of device 100 by 90 degrees or 180 degrees or 270 degrees,horizontal or vertical or diagonal swipe gesture, scroll gesture,zoom-in or zoom-out gestures, user operations on physical buttons orsliders or interface components of device 100 (e.g., volume interface,camera button, button for capturing an image or a video), or the like.

Correlator 131 may further detect correlations based on movement,spinning, rotation and/or acceleration of device 100, along one axis ortwo axes or three axes, that occur prior to or subsequent to a usergesture. For example, correlator 131 may detect that a first particularuser typically tilts the phone from being generally perpendicular to theground, to being generally parallel to the ground, immediately prior toperforming a zoom-out gesture (e.g., a “pinching” gesture with twofingers on touch-screen 105). Similarly, correlator 131 may detect thata second particular user typically rotates the phone counter-clockwise,immediately subsequent to performing a zoom-in gesture (e.g., spacingapart two fingers on touch-screen 105). In some implementations, forexample, a correlation may be detected while the user gesture isperformed, immediately before the user gesture is performed (e.g.,within 0.5 seconds prior to the user gesture), and/or immediately afterthe user gesture is performed (e.g., within 0.5 seconds subsequent tothe user gesture).

Optionally, correlator 131 may detect other suitable correlations, andmay take into account other types of readings or sensed data, forexample, data indicating a temperature or moisture level or sweat levelwhich may be associated with a user gesture, data indicating the amountof pressure or force applied by a user (e.g., when pressing ontouch-screen 105), or the like.

In a demonstrative example, a first user may typically scroll down withhis finger on touch-screen 105 while slightly rotating the mobile device100 around its longest axis; and a correlation may be identified betweenthe respective touch data and acceleration/orientation data, indicativeof the first user. In contrast, a second user may typically scroll downwhile maintaining the mobile device 100 non-rotating, or while rotatingmobile device 100 at a different direction or angle, or at a differentacceleration value, thereby allowing to identify a differentcorrelation, indicative of the second user.

Optionally, the present invention may identify, create and utilize afirst set of behavioral traits which correspond to the behavior of aparticular user when he is utilizing his mobile device in a firstholding scenario (e.g., when the user is holding the mobile device inhis hand), and a second (different) set of behavioral traits whichcorrespond to the behavior of that particular user when he is utilizinghis mobile device in a second holding scenario (e.g., when the mobiledevice is placed on a table or flat surface and the user operates themobile device without holding it). Accordingly, the present inventionmay create and utilize a behavioral profile for that user, which maycomprise multiple sub-profiles of behavioral traits that correspond tosuch multiple usage scenarios by the same (e.g., “genuine”) user. In asubsequent usage of the mobile device, the system may compare thebehavioral traits of the subsequent user, to each one (e.g., separately)of the pre-stored sets of behavioral traits (or behavioralsub-profiles), in order to detect or determine whether that subsequentuser is the “genuine” user operating in one of the known usagescenarios, or alternatively a fraudulent user or attacker. Similarly,the present invention may generate and/or utilize complex profiles thatmay comprise of sub-profiles or sets of traits (e.g., behavioral traits,physiological traits, motor control traits), such that each set orsub-profile may correspond to a particular usage scenario or aparticular holding scenario of the user; and a subsequent usage may becompared, separately, to each one of those sub-profiles (or sets oftraits) in order to determine user authenticity.

The terms “correlation”, “correlator”, “to correlate”, and similar orequivalent terms which may be used herein, are used for demonstrativepurpose only; they may include, for example, statistical correlation, orstatistically-significant correlation, or any other type of relation orindication or matching between two parameters or between groups ofvalues. In some embodiments, there need not be statistically-significantcorrelation between, for example, touch data and acceleration data, inorder to identify or extract unique user trait(s); but rather, there maybe other type of relation or matching between touch-data andacceleration data in order to determine such “correlation”.

In accordance with the present invention, mobile device 100 maycontinuously track and/or monitor the correlation between touch-data andacceleration/orientation data. Correlation values may be used todetermine user-specific traits, that are indicative of the user of themobile device 100, which may be regarded initially as the “genuine”user. Then, during subsequent usage of the mobile device 100,correlation between touch-data and acceleration/orientation data may betracked and identified, and may be compared to the correlationpreviously-determined for the genuine user, in order to confirm that acurrent user is indeed the genuine user, or in order to determine or toestimate that a current user is a non-genuine user.

In a demonstrative implementation, an application or a website may beaccessible through device 100 through an access control process or auser authentication process. Such application or website may be, forexample, an email account, a social network account, a video conferenceapplication, a chat application, an online banking application orwebsite, a securities trading application or website, an electroniccommerce account or website, or the like. The user may be prompted tocreate a new user account (e.g., define a username and password); andthen, or in parallel, user-specific traits may be captured throughpassive means and/or active means, which may be known to the user or maybe hidden from the user.

For example, a profile creation page or application may require the userto perform various touch operations (e.g., tapping, scrolling, dragging,or the like), and may capture touch data as well asacceleration/orientation data, which may then be correlated in order toidentify a biometric trait indicative of the user who is currentlycreating the profile, or who is otherwise believed to be a genuine user(e.g., based on password entry and/or responses to security questions orother challenge-response mechanisms). Optionally, an active challengemay be posed to the user, for example, by explicitly asking the user toperform one or more particular touch gestures on touch-screen 105,either as “hidden” challenges (in which the user is not aware that he isactively challenged for security purposes) or as non-hidden challenges(in which the user is advised that, as a security measure, he isrequired to perform certain touch gestures in order to extract biometrictraits).

Reference is made to FIG. 5, which is a flow-chart of a method inaccordance with some demonstrative embodiments of the present invention.The method may be implemented by a mobile electronic device, by one ormore hardware components and/or software modules of a mobile electronicdevice, by a system, or the like.

The method may include, for example, capturing at least one of touchdata, hovering data, motion data, gesture data (block 510).

The method may include, for example, capturing at least one ofacceleration data, gyroscope data, device orientation/rotation data,principal axes rotation data (e.g., normal axis or yaw, lateral axis orpitch, longitudinal axis or roll) (block 520).

The operations of block 520 may be performed simultaneously orconcurrently with, or in parallel to, the operations of block 510.

The method may include, for example, correlating or matching (block 530)between the data captured in block 510 and the data captured in block520.

The method may include, for example, extracting a user-specific trait(block 540) based on the correlating or matching of block 530. Theuser-specific trait may include, for example, one or more behavioraltraits, physiological traits, motor control traits, or otheruser-specific characteristics.

The method may include, for example, subsequently, confirming useridentity based on said user-specific trait (block 550).

Other suitable operations may be used in accordance with the presentinvention.

Some embodiments of the present invention may be utilized, or mayoperate, in conjunction with methods, algorithms, devices and/or systemswhich are described in PCT International Application NumberPCT/IL2011/000907, titled “Method and Device for Confirming ComputerEnd-User Identity”, published on Jun. 7, 2012 as InternationalPublication Number WO/2012/073233, which is hereby incorporated byreference in its entirety; and/or in U.S. patent application Ser. No.13/877,676, filed on Apr. 4, 2013, which is hereby incorporated byreference in its entirety.

In accordance with the present invention, correlation between touch-dataand acceleration/orientation data may be identified and/or checkedlocally in mobile device 100; or remotely, such as in a remote serverwhich may receive such data via a wireless communication link frommobile device 100; or by using other suitable architecture, for example,a hybrid architecture in which some operations may be performed locallyand other operations may be performed remotely. Accordingly, componentsor modules that are depicted, for demonstrative purposes, as beingincluded in mobile device 100, may be implemented at a remote server orwithin other suitable units. The present invention may be implemented ina stand-alone mobile device, such that data collection and processingmay be performed within device 100; or in a client-server architecture,such that device 100 may collect data and may wirelessly transmit thecollected data to a remote server for processing and analysis; or in a“cloud computing” architecture in which data is stored remotely and isalso processed remotely. Other suitable architectures may be used, todeploy a system in which a particular mobile device “knows” orrecognizes its genuine user, or, to deploy a system in which aparticular application or website “know” or recognize a genuine user,based on the above-mentioned correlations.

In a demonstrative experiment in accordance with the present invention,multiple participants were asked to hold a particular mobile device (aniPad tablet), to drag (with a finger) a displayed green circle towards adisplayed red target, and then to release the dragged item once itreached the red target. Accelerometer data and touch data were collectedwhile performing the requested operations.

The experiment measured the touch and release signals, as well asaccelerometer measurements; and then triggered the acceleration dataaccording to the touch time. FIG. 2 depicts three graphs 201-203, whichdemonstrate acceleration as a function of time over three separate axes,thereby demonstrating at least two identifying characteristics which maybe used as a user-specific trait. As a first identifying characteristic,the phasic level (observed at the X axis) may have different values fordifferent people, corresponding to different posture of the mobiledevice. As a second identifying characteristic, the transient shape oncethe device is clicked (observed at the Z axis) may have different valuesfor different people. This data may be transformed or analyzed, forexample, by using dimension reduction techniques (e.g.,kernel-principle-component-analysis), thereby demonstrating thebiometric capability of synergizing between touch data and accelerationdata.

Reference is made to FIG. 3, which demonstrates a graph 300 of the mainaxes of the dimension-reduced space of the accelerometer reaction totapping. Each small item in graph 300 represents one trial, and eachshape or character in graph 300 (e.g., circle, square, diamond,triangle) represents a different user. This drawing demonstratesidentifiable clusters 301-309 of trials, each such cluster correspondingto a different user.

In certain scenarios, posture data (e.g., phasic response) may beneglected or may not be available, for example, if the mobile device isoperated while being placed on a table or a flat surface and is nothand-held by the user. In such scenarios, only the device's kinematicsduring taps may be taken into account, and still the present inventionmay capture sufficient information for biometric functions. Reference ismade to FIG. 4 which illustrates a graph 400 depicting the featurespace, where each dot represents a trial; greyed circles representtrials performed by one particular user, and black circles representtrials performed by the other participants. This drawing demonstratesdimension reduction when only the device's kinematics are taken intoaccount, showing that, still, sufficient significant biometricinformation may be captured and determined.

The present invention may be used in order to automatically identifythat a user (e.g., an attacker or a “fraudster”) is attempting to poseas (or impersonate, or “spoof”) another user (e.g., the “real” user orthe genuine user). In accordance with the present invention, theattacker would need to carefully and correctly imitate the exactaccelerometer response for tapping (or for other suitable touch-screenoperations, such as scrolling, dragging, releasing), taking into accountthe particular kinematics properties of the genuine user; and suchimitation may be extremely difficult and unlikely, or even impossible,for most attackers.

The present invention may utilize signal processing and/or machinelearning techniques, in order to build or generate a template model or aprofile which corresponds to the genuine user; and then comparesubsequent instance(s) or sample(s) to the pre-built (and locallystored, or remotely stored) model or profile. If the subsequent samplesare consistent with the pre-built model or profile, then a first outputscore may be generated (e.g., having a high value in a predefinednumeric range, such as a value of 98 on a scale of 0 to 100); whereas,if the subsequent samples are inconsistent with the pre-built model orprofile, then a second output score may be generated (e.g., having alower value on the predefined numeric range, such as a value of 34 onthe scale of 0 to 100). In some implementations, an output score greaterthan a threshold value may be used (alone, or in combination with otherbiometric traits and/or other authentication measures) as an indicationthat the current user is the genuine user; whereas an output score lowerthan the threshold value may be used (alone, or in combination withother biometric traits and/or other authentication measures) as anindication that the current user is not the genuine user.

The present invention may further be used to differentiate ordistinguish between the genuine (human) user, and a robot or amachine-operable module or function (e.g., implemented as a computervirus, a Trojan module, a cyber-weapon, or other malware) which attemptsto automatically imitate or emulate or simulate movement of a cursor orother interaction with a touch-screen. For example, false identitycreated by automated malware may be detected by the present invention assuch automated malware may lack the characterization of human (e.g.,manual) behavior, such as the touch features (e.g., speed, pressure)and/or its accelerometer correlated measurements.

The present invention may operate and may provide an efficient biometricor user-authentication modality, without capturing, storing, orotherwise identifying any Personally Identifiable Information (PII). Forexample, the present invention may be used to distinguish between agenuine user and a fraudster, without knowing any PPI of the genuineuser and/or of the fraudster.

The present invention may detect correlations and extract user-specifictraits based on passive data collection and/or based on activechallenges. In passive data collection, the mobile device may detectthat the user is performing a particular operation (e.g., a verticalscroll gesture), and may further detect that performing this gestureaffects in a user-specific way the acceleration and/or theorientation/rotation of the mobile device. In an active challenge, themobile device (or an application or process thereof) may activelypresent a challenge to the user, such as, a requirement to the user toperform horizontal scrolling, in order to capture data and detectuser-specific correlation(s). The active challenge may be hidden or maybe unknown to the user, for example, implemented by creating a GraphicalUser Interface (GUI) that requires the button to scroll in order toreach a “submit” button or a “next” button or a “continue” button,thereby “forcing” the user to unknowingly perform a particularuser-gesture which may be useful for correlation detection or forextraction of user-specific traits, as described. Alternatively, theactive challenge may be known to the user, and may be presented to theuser as an additional security feature; for example, by requesting theuser to drag and drop an on-screen object from a first point to a secondpoint, as an action that may be taken into account for confirming useridentity.

Some embodiments of the present invention may be implemented, forexample, as a built-in or integrated security feature which may be acomponent or a module of a mobile device, or may be a downloadable orinstall-able application or module, or plug-in or extension; or as amodule of a web-site or web-page, or of a client-server system or a“cloud computing” system; or as machine-readable medium or article ormemory unit able to store instructions and/or code which, when executedby the mobile device or by other suitable machine (e.g., a remoteserver, or a processor or a computer) cause such machine to perform themethod(s) and/or operations described herein. Some units, components ormodules, that are shown in FIG. 1 for demonstrative purposes ascomprised within mobile device 100, may be implemented externally tomobile device 100, may be implemented in a remote server, a web server,a website or webpage, a “cloud computing” server or database, aclient/server system, a distributed system, a peer-to-peer network orsystem, or the like.

In some embodiments of the present invention, the analysis orcorrelation or matching (e.g., between accelerometer/gyroscope data, andtouch-data or hovering data or other user-gesture data) may belocation-based and/or application-based, or may otherwise take intoaccount a geographical location or geo-spatial location of the mobiledevice or the application(s) being used or that are installed on thedevice. In a demonstrative example, a suitable module (e.g., alocation-aware module or location-determining module) in the mobiledevice may determine the current location of the mobile device, based onGPS data or Wi-Fi data or cellular triangulation data or mobile networkcell data or other location-identification techniques. The mobile phonemay then utilize a suitable module (e.g., a correlator or matchingmodule between location and user-specific behavioral usage traits) inorder to deduce or determine, for example: that when the user utilizeshis mobile device in a first location (e.g., in his office), then themobile phone is typically placed horizontally on a flat surface (e.g., atable); that when the user utilizes his mobile phone in a secondlocation or type of location (e.g., outdoor, on the street, in thepark), then the mobile phone is typically held by the hand of the userat a slanted angle or diagonally (e.g., at approximately 45 to 60degrees relative to the ground); that when the user utilizes his mobilephone in a third location or type of location (e.g., at a Point-Of-Sale(POS) terminal or register or cashier, at a supermarket or a retailstore), then the mobile phone is typically held generally horizontallyby the hand of the user (e.g., generally parallel to the ground); thatwhen the user utilizes his mobile phone in a fourth location or type oflocation (e.g., at an Automatic Teller Machine (ATM) or a vendingmachine), then the mobile phone is typically held generally verticallyby the hand of the user (e.g., at an angle of approximately 90 degrees,or between 80 to 100 degrees, relative to the ground); or the like.These determinations may be location-based or location-aware, therebytriangulating or crossing among three dimensions, namely, behavioraluser-specific traits (e.g., holding the phone diagonally), gesture data(e.g., performing a scroll-down gesture), and location data (e.g., whenutilizing the phone at a retailer); and such determinations may be partof the user-specific profile of that user. In a subsequent usage of themobile device, similar determinations may be made, in order to analyzewhether or not a current user is indeed the same user as in previoususage session(s) or is a “genuine” user. In a demonstrative example,this three-prone approach may raise an alert if, for example, typicallythe user of the mobile device holds his mobile device horizontally whenperforming a scroll-operation at a Point of Sale terminal; and in asubsequent usage session of the mobile device, a user holds that phonevertically when performing a scroll-operation at such Point of Saleterminal, thereby indicating that the subsequent user may not be thegenuine or authorized user of the mobile device. In some embodiments,these multi-prone determinations may further be augmented with, ormatched or correlated with, application-specific data orapplication-specific determinations, in order to improve the tailoringof the behavioral traits to the specific user. For example, the mobiledevice may differentiate and determine that the genuine user typicallyholds the phone vertically (e.g., anywhere, or in a particular locationor type of location) when utilizing the camera application of the mobiledevice, but typically holds the phone horizontally (e.g., anywhere, orin that particular location or type of location) when utilizing theaddress book application of the mobile device; and these user-specifictraits may be extracted and subsequently compared to data captured in asubsequent usage session of that mobile device, to authenticate useridentity.

The present invention may be used in conjunction with various suitabledevices and systems, for example, various devices that have atouch-screen; an ATM; a kiosk machine or vending machine that has atouch-screen; a touch-keyboard; a system that utilizes Augmented Reality(AR) components or AR glasses (e.g., Google Glass); a device or systemthat may detect hovering gestures that do not necessarily touch on thescreen or touch-screen; a hovering screen; a system or device thatutilize brainwave analysis or brainwave control in which the user'sbrainwaves are captured or read and the user's brain may directlycontrol an application on the mobile device; and/or other suitabledevices or systems.

Some embodiments may provide a method for confirming identity of a userof an electronic device, the method comprising: receiving touch datafrom a touch-screen of the electronic device; receiving accelerationdata from an accelerometer of the electronic device; determining arelation between the touch data and the acceleration data; based on saidrelation between the touch data and the acceleration data, generating auser-specific trait indicative of said user of said electronic device;storing, either locally within said electronic device or on a remoteserver, a reference value of said user-specific trait; in a subsequentusage session, generating and storing a current value of theuser-specific trait indicating relation between touch data andacceleration data; and based on a comparison process between (A) thecurrent value of the user-specific trait that was generated, and (B) thereference value of the user-specific trait that was previouslygenerated, determining whether or not a current user of the smartphoneis an authorized user of the smartphone.

In some embodiments, said touch data comprises non-tactile touch dataindicating a hovering user gesture in proximity to said touch-screen ofsaid electronic device.

In some embodiments, the method comprises: receiving gyroscope data froma gyroscope of the electronic device; determining a relation between thetouch data and the gyroscope data and the acceleration data; based onsaid relation between the touch data and the gyroscope data and theacceleration data, generating another user-specific trait indicative ofsaid user.

In some embodiments, the method comprises: based on the relation betweenthe touch data and the acceleration data, (A) determining that a firstphysiological region of said user moves when a particular gesture isperformed, and (B) determining that a second physiological region ofsaid user does not move when said particular gesture is performed; basedon said two determining operations, differentiating among multipleusers.

In some embodiments, the method comprises: determining an offset ofholding said electronic device in a hand of said user, wherein theoffset comprises an offset selected from the group consisting of: theelectronic device being held with a palm area of the hand, and theelectronic device being held with a fingers area of the hand; based onsaid offset of holding the electronic device in the hand,differentiating among multiple users.

In some embodiments, the method comprises: determining whether (A) thesame hand of the user is utilized for both holding the electronic deviceand tapping the touch-screen of the electronic device, or (B) a firsthand of the user is utilized for holding the electronic device and asecond hand of the user is utilized for tapping the touch-screen of theelectronic device; based on said determining, differentiating amongmultiple users.

In some embodiments, the method comprises: constructing a user-specificprofile based on said touch data and said acceleration data, wherein theconstructing is performed over a pre-defined time-period; dynamicallyshortening the pre-defined time period for constructing saiduser-specific profile if one or more identified traits of said user aredistinctive.

In some embodiments, the method comprises: constructing a user-specificprofile based on said touch data and said acceleration data, wherein theconstructing is performed within a constraint selected from the groupconsisting of: (A) a pre-defined time-period, and (B) a pre-definednumber of user interactions; dynamically modifying said constraint forconstructing said user-specific profile, based on distinctiveness of oneor more traits of said user; storing a flag indicating whether saiduser-specific profile is either (i) under construction, or (ii) fullyconstructed.

In some embodiments, the method comprises: constructing a user-specificprofile which indicates that for a user-gesture that is performed at aparticular geometric place of the touch-screen of said electronicdevice, a first body part of the user is moving while a second body partof the user is at rest; based on said user-specific profile,differentiating among multiple users.

In some embodiments, the method comprises: constructing a user-specificprofile which indicates that for a scrolling gesture that is performedon the touch-screen of said electronic device, a first hand-region ofthe user is moving while a second hand-region of the user is at rest;based on said user-specific profile, differentiating among multipleusers.

In some embodiments, the method comprises: analyzing touch-data of aswipe gesture performed by the user on the touch-screen of saidelectronic device, to determine an estimated width of a finger of saiduser; constructing a user-specific profile which comprises saidestimated width of the finger of the user; based on said user-specificprofile, differentiating among multiple users.

In some embodiments, the method comprises: analyzing touch-data of aswipe gesture performed by the user on the touch-screen of saidelectronic device, to determine an estimated width of a finger of saiduser; constructing a user-specific profile which comprises saidestimated width of the finger of the user; based on said user-specificprofile, differentiating among multiple users.

In some embodiments, the method comprises: analyzing touch-data of acircular swipe gesture performed by the user on the touch-screen of saidelectronic device, to determine an estimated distance between (A) a tipof a swiping finger of a hand of said user, and (B) a palm of said handof said user; constructing a user-specific profile which comprises saidestimated distance between the tip of the swiping finger and the palm ofthe hand; based on said user-specific profile, differentiating amongmultiple users.

In some embodiments, the method comprises: analyzing touch-data ofgenerally-straight swipe gestures performed by user on the touch-screenof said electronic device; determining that a first user typicallyrotates the electronic device clockwise while performinggenerally-straight swipe gestures; determining that a second usertypically rotates the electronic device counter-clockwise whileperforming generally-straight swipe gestures; based on saiddeterminations, differentiating among said first and second users.

In some embodiments, the method comprises: analyzing said touch data andsaid acceleration data of said electronic device, to determine a levelof shakiness of the electronic device while the user operates saidelectronic device; analyzing said touch data and said acceleration dataof said electronic device, to determine an effect, of a performeduser-gesture, on said level of shakiness of the electronic device;constructing a user-specific profile which comprises an indication ofthe effect of the performed user-gesture on the level of shakiness ofthe electronic device; based on said user-specific profile,differentiating among multiple users.

In some embodiments, the method comprises: analyzing said touch data andsaid acceleration data of said electronic device, to determine thatimmediately prior to performing an on-screen zoom gesture, the user ofthe electronic device modifies a tilt angle of the electronic devicerelative to ground; constructing a user-specific profile which comprisesan indication that immediately prior to performing on-screen zoomgestures, the user of the electronic device modifies the tilt angle ofthe electronic device relative to ground; based on said user-specificprofile, differentiating among multiple users.

In some embodiments, the method comprises: sensing by said electronicdevice a sweat level of the user while the user performs a gesture onsaid electronic device; determining a relation between the sensed sweatlevel of the user and at least one of: said touch data of the electronicdevice, and said acceleration data of said electronic device; based onsaid relation, differentiating among multiple users.

In some embodiments, the method comprises: sensing by said smartphone atemperature of a body part of the user while the user performs a gestureon said electronic device; determining a relation between the sensedtemperature and at least one of: said touch data of the electronicdevice, and said acceleration data of said electronic device; based onsaid relation, differentiating among multiple users.

In some embodiments, the method comprises: sensing by said electronicdevice an amount of pressure of a body part of the user while the userperforms a gesture on said electronic device; determining a relationbetween the sensed amount of pressure and at least one of: said touchdata of the electronic device, and said acceleration data of saidelectronic device; based on said relation, differentiating amongmultiple users.

In some embodiments, the method comprises: determining a currentlocation of the electronic device; determining a relation among: (A) thecurrent location of the electronic device, and (B) said touch data ofthe electronic device, and (C) said acceleration data of the electronicdevice; based on said relation, differentiating among multiple users.

In some embodiments, the method comprises: determining geographiclocation of the electronic device; determining a relation among: (A) thecurrent location of the electronic device, and (B) said touch data ofthe electronic device, and (C) said acceleration data of the electronicdevice; based on said relation, (a) determining that a first user,typically places the electronic device horizontally on a flat surfacewhen utilizing the electronic device in a first geographic location, and(b) determining that said first user, typically holds the electronicdevice slanted relative to the ground when utilizing the electronicdevice in a second geographic location; based on said determinations,differentiating among the first user and another user.

In some embodiments, the method comprises: determining a currently-usedapplication of the electronic device, that the user is currentlyutilizing on said electronic device; determining a relation among: (A)the currently-used application of the electronic device, and (B) saidtouch data of the electronic device, and (C) said acceleration data ofsaid electronic device; based on said relation, differentiating amongmultiple users.

In some embodiments, the method comprises: determining a currently-usedapplication of the electronic device, that the user is currentlyutilizing on said electronic device; determining a relation among: (A)the currently-used application of the electronic device, and (B) saidtouch data of the electronic device, and (C) said acceleration data ofthe electronic device; based on said relation, (a) determining that afirst user typically holds the electronic device vertically whenutilizing a first particular application of the electronic device, and(b) determining that said first user typically holds the electronicdevice slanted relative to the ground when utilizing a second particularapplication of the electronic device; based on said determinations,differentiating among multiple users.

In some embodiments, the method comprises: determining whether a currentlocation of the electronic device is outdoors or indoors; determining arelation among: (A) the current location of the electronic device beingeither outdoors or indoors, and (B) said touch data of the electronicdevice, and (C) said acceleration data of said electronic device; basedon said relation, differentiating among multiple users.

The present invention provides a method for confirming identity of auser of a mobile electronic device, the method including: receivingtouch data from a touch-screen of the mobile electronic device;receiving acceleration data from an accelerometer of the mobileelectronic device; correlating between the touch data and theacceleration data; based on the correlating, generating a user-specifictrait indicative of said user. The method further includes storing areference value of the user-specific trait, indicative of said user; ina subsequent usage session of the mobile electronic device, generating acurrent value of the user-specific trait correlating between touch dataand acceleration data; and based on a comparison between the currentvalue of the user-specific trait and the reference value of theuser-specific trait, determining whether or not a current user of themobile electronic device is an authorized user of the mobile electronicdevice.

Some embodiments of the present invention may enable the system toensure and/or to confirm “Liveness” of the user, of an image or video ofthe user, or a self-image or self-video of the user, that are submittedby a user or by an electronic device having a camera or imager, forpurposes of user authentication or user identity determination, or forpurposes of authorizing or blocking access of a user to a local orremote computerized service and/or to the electronic device itself;and/or to prevent, avoid, eliminate, block and/or mitigate a problem ora fraud-attempt in which an attacker or a non-legitimate user or animpostor attempts to perform “spoofing” by submitting to the system (orthe electronic device itself, or to a local or remote authenticationmodule) a non-fresh image or video, or a non-freshly-taken image orvideo, or a previously-recorded image or video or self-image orself-video, or a recording or a capturing of the user or of an image ofthe user of a video of the user, or of a play-back or a replay or are-broadcast attempt that attempts to re-submit to the authenticationmodule a previously-captured image or video or an attempt to submit animage or video that lack Liveness and/or that lack Freshness; as well asto provide Liveness Detection and/or Freshness Detection for suchuser-authentication system.

It is also clarified that although portions of the discussion herein mayrelate, for demonstrative purposes, to utilization of an image or animage-stream or a video-sequence for user-authentication purposes, thepresent invention may also be used in conjunction withuser-authentication systems and methods and modules that utilize otherbiometric traits and/or user-specific traits, for example, user speech,user utterance, user audio sample, user audible sounds or words, acombination of audio and video, a combination of audio and an image, acombination of audio and a set of images, a fingerprint of a user, acombination of fingerprint with image and/or video and/or audio, and/orother suitable biometric features and/or combination of biometricfeatures.

The present invention may provide methods, devices, and systems for userauthenticating a user, based on an image or photograph or video of theuser; and for enabling a log-in process or a sign-in process, to adevice or to a local computerized service or to a remote computerizedservice, based on an image or photograph or video of the user; and forauthorizing (or approving, or denying, or blocking) access of a user toa computerized service or to an electronic device, based on an image orphotograph or video of the user; and for authorizing a user to “unlock”an electronic device, or to switch an electronic device from “sleepmode” or “standby mode” to fully operational mode, or from beingdeactivated to being activated, or from being partially-operational tobeing fully-operational, based on an image or photograph or video of theuser. The present invention may further be utilized in other contexts,for example, to authorize or to block physical access of a user into abuilding or through a physical gate or entrance or exit; to authorize orto block access of a user to a vehicle, or to authorize or to block anattempt of the user to ignite or to start or to drive a vehicle, or thelike.

In accordance with the present invention, a freshly-captured image or avideo or a photograph of the user may be utilized foruser-authentication purposes, in order to authenticate the user to alocal device, a remote device, a local service and/or a remote service.For example, a system may firstly or initially capture or receive orobtain a Trusted Image of the user; for example, an image or a video ofthe user that the user takes in the presence of a third party (e.g., inthe presence of a bank clerk, when the computerized service is a bankingwebsite or a banking application), or an image or a video of the userthat the user takes upon initial creation of a user account, or uponopening a new account, or during a session that is known to beauthenticated (e.g., during a usage session that was authorized via atwo-step authentication process or via a two-factor authenticationprocess). The Trusted Image, or other such “Reference Image”, may besecurely stored locally (e.g., in the electronic device, smartphone,tablet, laptop computer, or the like), and/or may be securely storedremotely (e.g., in a remote server, web server, application server,“cloud computing” server, or the like).

Subsequently, when the user requests to re-access that service or thatelectronic device, the user is requested to capture a Fresh Image ofhimself, via an imager or a camera of an electronic device beingutilized by the user (e.g., smartphone, tablet). The Fresh Image iscompared to the Trusted Image, or is analyzed in relation to it. If thetwo images are generally matching each other, then the user isauthenticated; otherwise, the authentication attempt is blocked orrejected.

In some embodiments, the user may be required to perform one or moregestures or modifications or actions, as part of the authenticationprocess, in order to “prove” to the authentication system that the useris indeed capturing a Fresh Image using the camera, and not, forexample, pointing the camera of the device towards a previously-takenimage of the user; for example, in order to stop an attacker who obtainsa still image of the legitimate user, and then attempts to merely pointthe camera of the electronic device towards such still image. Forexample, the system may request the user to Blink one eye, or two eyes,or to perform a sequence of Eye Blinks, or to perform other gestures(e.g., touch his nose with one finger), in order to utilize imageanalysis and/or video analysis processes, to ensure that the capturedimage(s) or video indeed depict the required gestures or actions;thereby preventing an attacker from merely pointing the camera towards astill image, or towards a previously-captured image or video of thelegitimate user.

Applicants have realized that conventional systems that attempt toutilize a Fresh Image of a user as means for user authentication, sufferfrom one or more of the following problems or disadvantages: (1) manyusers utilize a mobile electronic device, such as smartphone or tablet,which has a limited power source (e.g., internal rechargeable battery),and utilization of the camera consumes significant power from theinternal power source electronic device, thereby depleting the powersource rapidly, and thereby causing resentment of users fromcontinuously utilizing the device's camera for user-authenticationpurposes. (2) Any third-party application or software module, thatperforms user authentication based on a freshly-captured image or videoof the user, that is installed on the mobile device by any third-partyother than the original manufacturer of the electronic device, may beinherently insecure and may be compromised; for example, a maliciouscode or Trojan or computerized virus or a Remote Access Trojan (RAT) maybe used by an attacker in order to gain access to the electronic deviceand then capture or obtain an image or a “selfie” or a “selfy” image ofthe user, or to obtain or copy an image or a video of the user from theGallery of images or videos that is stored locally within the electronicdevice, thereby compromising the user-authentication process, since theattacker may then utilize the stolen image or the stolen video of theuser to authenticate, pretending to be the legitimate user. (3) Athird-party application that activates and/or utilizes the camera of thedevice, for a prolonged time or from prolonged periods of times, orcontinuously for user authentication purposes, create a privacy risk tothe user and/or a security risk; since such application may become atool or a backdoor for hackers or attackers that may continuouslycapture images and/or videos of the user, without the user's knowledgeor consent. (4) Continuous analysis of image(s) and/or video-sequences,for purposes of user authentication, is a process that consumessignificant resources (e.g., memory resources, processing resources,power/battery resources), thereby depleting or draining the resources ofthe electronic device and/or degrading the quality of operation of theelectronic device (e.g., making the device slower, less responsive).

The Applicants have realized that utilization of an image or video ofthe user, or a “selfie” photograph (self-taken photograph) of the user,for purposes of user authentication, thus suffers from multiple problemsor disadvantages: firstly, it attempts to utilize a “secret” or“confidential data” of the user, which is actually available to thegeneral public since an attacker may capture an image or video of thelegitimate user, and/or which is available to a skilled hacker byaccessing images or videos of the legitimate user (e.g., on the deviceitself; in a “cloud” repository of images or videos of the user; on aSocial Network website or service in which the user posts or shares hisvideos or images, or in which friends or relatives of the user post orshare their own images or videos that contain the legitimate user withinthem). The security risk increases when the authentication module is notbuilt-in within the electronic device, but rather is a third-partyapplication or a part thereof. Additionally or alternatively, theimage-based or video-based user authentication, which appears at firstglance to be a “convenient” or a “rapid” process of authentication thatmany users may like, actually drains or depletes resources of the mobiledevice (e.g., battery resources, processing resources, memoryresources), thereby creating resentment of some users to continuous orfrequency utilization of such user-authentication process via aresources-limited electronic device.

The present invention may strengthen the user-authentication process, byincluding in the authentication process also a sub-process in which thesystem takes into account “spatial properties” of the device, such as,accelerometer data, gyroscope data, device orientation data, devicepositioning data, device movement data, device tilt data (e.g., is thedevice generally horizontal, or generally vertical, or slanted atapproximately 30 degrees), device traits that characterize the devicewhen the user captures an image or a video, and/or user-specific traitsthat characterize the user when he captures an image or a video forauthentication purposes; and/or the modification or the change in valuesof such “spatial properties”; and/or such traits and such data (or, thechange in values of such spatial properties) during a time period (e.g.,1 or 2 or 3 or 5 or 10 or K seconds) immediately prior to capturing theimage or video for authentication purposes; and/or such traits and suchdata during a time period (e.g., 1 or 2 or 3 or 5 or 10 or K seconds)immediately after capturing the image or video for authenticationpurposes. Such additional data and traits may be correlated with theother user-authentication data (e.g., the actual content of the image orthe video), in order to improve and strengthen the user-authenticationprocess.

The system of the present invention may correlate between: (I) contentof an image or a video, or a “freshly captured” image or video, or a“selfie” photograph or video, or an indicator that a self-image orself-video is being captured or has recently been captured; with (II)accelerometer data and/or gyroscope data and/or device-orientation dataand/or device-tilt data, in order to further authenticate the user or toraise a flag that a possible attack is being performed by an attackerinstead of the legitimate user.

In a first demonstrative example, the system may monitor the physicaltraits of the electronic device (e.g., acceleration, orientation,tilting, position, velocity, shaking, or the like), during the time inwhich the self-image or self-video are captured (and/or immediatelybefore, and/or immediately after); and may determine that the physicaltraits of the electronic device do not match a physical process ofcapturing a video or capturing an image. For example, the camera of theelectronic device may “see” or may capture an image or a video thatappears to be of the user; whereas the device itself reports being atrest and/or not moving, and/or not sufficiently moving in correlationwith capturing of an image or a video; thereby indicating that anattacker may be controlling the authentication process, for example, byattempting to submit a previously-captured image or video of thelegitimate user. The attacker or possible-attack may be determined orestimated; and the authentication of the user to the device (or to thecomputerized service, which may be local or remote) is blocked orrejected.

In a second demonstrative example, the system may determine that userAdam, who from time to time authenticates by using a self-taken image ofhimself, typically or always captures his own image by (a) lifting hissmartphone to be at approximately chest level, and (b) tilting hissmartphone to be at an angle of approximately 45 degrees relative to theground. However, in a subsequent user-authentication attempt, the systemmay monitor the device traits, and may determine that (A) the device islifted to be at eye level, and/or (B) the device is capturing the imagewhile the device is generally perpendicular to the ground. The systemmay determine that these fresh traits that characterize the freshcapturing of the self-image or self-video, do not match the historic orprevious traits that characterized the capturing of the image or videoin prior authentication sessions, or in when a Trusted or a Referenceimage (or video) was captured; thereby enabling the system to determineor to estimate that an attacker is attempting to authenticate, and notthe legitimate user.

In a third example, the electronic device may request the user to take aself-image, and may actually capture a stream or set of images or avideo sequence (e.g., including images before, during and after themoment in which the user pushes a “snapshot” button). The electronicdevice may determine that the sequence of multiple frames or multipleimages, does not show any “shaking movement”; whereas accelerometer dataand/or gyroscope data indicate that the device was “shaken” or “shaking”during that time period or a portion thereof. The system may thusdetermine or estimate that a fraudulent log-in attempt or a fraudulentauthentication attempt is being carried out, and may block or reject theattacker.

The present invention may thus correlate between self-taken image(s)and/or self-taken video, or “freshly captured” image(s) or video, andfresh data and traits of the electronic device during (and/or before,and/or after) capturing such image or video, in order to verify: (a)that the image or video was indeed freshly captured in real time, basedon accelerator data, gyroscope data, device orientation data, or thelike, rather than the image or the video being submitted by an attacker;(b) that the image (or video) and the accelerometer data (or gyroscopedata, or other device data) match (or do not contradict each other) fromorientation perspective or from gyroscope perspective, or the like; forexample, “shake” events of the image or the video are indeed correlatingto actual “shake” movements of the electronic device; image or videoshakes are correlated with accelerometer data; that the orientation andgyroscope data are matching to the camera position in relation to theuser; that hands shaking are the same, that video movements are matchingwith gyroscope and accelerometer data, or the like.

In some embodiments, the system of the present invention may provide apower-saving and/or resources-saving user authentication process thatutilizes a self-image or a self-video of the user; for example, byselectively activating the imager or the camera only in the beginning of(or, only during an initial phase of) the user-authentication process;then, deactivating or turning-off the camera or imager of the device, toconserve power; and subsequently, to re-activate or turn-on the imageror camera again, only if device traits (e.g., accelerometer data,gyroscope data, device-orientation data) indicate that a risk isestimated or that an additional or extended validation of the useridentity is required.

For example, instead of continuous activation of the camera or imagerduring an entirety of the user authentication process (e.g., spanning aperiod of ten seconds), the present invention may utilize a three-stepmethod.

(A) Firstly, the camera is turned-on for a shortened time-period (e.g.,only two seconds; by a camera activation unit or module), therebycapturing a short sequence of images or video, while the device alsocollects or senses or monitors device-traits (e.g., acceleration,gyroscope data, device orientation, device tilt, or the like).

(B) Then, the camera is turned-off or deactivated (by a cameradeactivation unit or module), to conserve power or to save power, or toreduce power consumption; while the electronic device continues tosense, monitor and collect device-traits (e.g., acceleration, gyroscopedata, device orientation, device tilt, or the like).

(C) Then, the method proceeds by branching into one of two branches ofoperations. In a first branch (C1), immediate analysis of the devicetraits, indicates that the device traits indeed correlate with the image(or video) that were captured; and therefore no further imaging isrequired. Alternatively, in a second branch (C2), immediate analysis ofthe device traits, indicates that the device traits do not correlatewith (or do not match) the image (or video) that were allegedlycaptured, and the camera is again activated (or turned on) in order tocapture additional frame(s) of the user for further user-authenticationpurposes.

In accordance with this method, the user authentication begins with facerecognition in correlation with accelerometer/gyroscope data (or otherspatial properties of the device, as sensed or measured via thesensor(s) of the device; and then continues without an activated cameraand only with the accelerometer/gyroscope activated, in order to savebattery and save power consumption, while the system “knows” that indeedthe same user is being authenticated. The accelerometer/gyroscope of theelectronic device thus operate as a “continuation” means to thefreshly-captured video (or sequence of frames or images), but therebyconsuming significantly less power relative to an active camera orimager; and only if the device or the system estimates a risk, based onthe accelerometer/gyroscope data, then the camera will again beactivated for further validation or for final validation of the useridentity

The method and system of the present invention may be utilized with avariety of devices, and/or for authenticating a user to a variety ofpurposes; for example, for accessing a smartphone or tablet orsmart-watch or laptop computer or desktop computer; for accessing anInternet of Things (IoT) device or appliance, or an Internet connectedappliance or device; for accessing a website or a web-based service, amobile application, an “app”, a computerized service, a “cloudcomputing” service, a remote service, a secured service that requiresauthentication (e.g., email account, web-mail account, social networkaccount, online banking account, online brokerage account); foraccessing or traversing or entering or exiting a physical gate or dooror entrance or exit (e.g., of a building, an office, a vehicle); forigniting or starting or driving a vehicle; and/or for other suitabletasks or systems in which user authentication is utilizes as a conditionfor granting or blocking access.

Some embodiments of the present invention may perform, utilize,determine and/or estimate, for example, relation or correlation or otherassociation between: (a) one or more behavioral parameters or traits, orone or more device-specific traits or user-specific traits that may besensed or measured or estimated when (or during, or before, or after)the user handles or operates the electronic device (e.g., accelerationof the electronic device, information sensed by accelerometer(s) and/orgyroscope(s) of the electronic device, slanting or tilting of theelectronic device, or the like), or user-specific behavioral traits(e.g., user Adam typically lifts his smartphone vertically beforecapturing a self-image; user Bob typically spins his smartphonecounter-clockwise immediately before or immediately after taking aself-image); and, (b) the actual content of the biometricuser-identifying or user-authenticating trait, such as, self-image orimage of the user, self-video or vide of the user, audio or speech orvoice of the user, fingerprint of the user, or the like.

The monitoring of these two groups or sets of parameters or information,and the estimation or the detection of a relation or a correlation or anassociation between the two groups or sets, may be utilized: (A) toensure Liveness or to confirm Liveness, and/or (B) to eliminate orreduce “spoofing” attacks or other attacks or fraud attempts, and/or (C)to increase the reliability and/or the Freshness of the biometriccontent that is relied upon (e.g., the content of the image or video oraudio or fingerprint), and/or (D) to save or conserve batteryconsumption or power consumption, particularly in an authenticationsystem that utilizes such biometric authentication frequently (e.g.,several times per hour or per day) and/or continuously (e.g., byselectively activating the imager or other biometric sensor onlypart-time during an initial stage of the user-authentication process,and utilizing other sensors such as accelerometer and gyroscopes, whichare less power-consuming, to provide assistive measurements which may becorrelated or associated as described above).

In some embodiments, the electronic device may comprise one or morefingerprint scanner(s) and/or fingerprint reader(s), which may belocated at the front side (e.g., screen size) of the device and/or atthe rear side (e.g., back side) of the device. The fingerprint scannermay capture a fingerprint scan of a user, which may be used by theelectronic device, or by a remote server, for user-authenticationpurposes, or as a user-authentication factor (e.g., by itself, or aspart of a multi-factor user-authentication processes). For example, areference fingerprint scan of the user may be initially (orperiodically) captured; and it may be stored (or, its digitalrepresentation or hashed value or encrypted version may be stored),locally within storage or secure storage of the electronic device and/orremotely at a remote server to which the user authenticates. In asubsequent log-in or log-on or user-authentication session, the user isrequested to capture and/or to submit, for authentication purposes, afresh or freshly-captured fingerprint scan, which is then compared to(or analyzed for similarity relative to) the reference fingerprint scan.If the freshly-captured fingerprint scan, is sufficiently similar to thepreviously-stored reference fingerprint scan (or, if their hashed valuesare sufficiently similar), taking into account one or more thresholdvalues of similarity and/or range of values that indicate similarity,then the user is authenticated.

Applicants have realized that utilization of such fingerprint scan(s)may suffer from a “liveness” problem, in which an attacker or animpostor may submit a “lifted” fingerprint or an imitation fingerprint,or a digital representation thereof, as an attempt to authenticate asthe legitimate user. Accordingly, the present invention may checkwhether, during the submission of the fresh fingerprint scan, and/orshortly before such submission (e.g., in the K seconds or millisecondsthat immediately precede the submission of the “fresh” fingerprint scanto the authentication module or process; for example, K being 1 secondor 2 seconds or 3 seconds), and/or shortly after such submission, theelectronic device has or has not sensed, via its sensors (e.g.,accelerometer(s), gyroscope(s), compass unit(s), other sensors),geo-spatial characteristics or device characteristics that correspond toa user touching or engaging with such fingerprint scanner; such as,change in acceleration or deceleration of the device, change in deviceorientation or slanting angle or tilt, change in device velocity orspeed of movement in space, spinning or rotation of the device, or thelike. Detection of such geo-spatial changes may indicate that thecurrently-submitted fingerprint scan is indeed “freshly captured”;whereas, lack of such geo-spatial changes, or detection of insufficientor small spatial changes (e.g., smaller than a pre-defined thresholdvalue), may indicate (by itself, and/or with other factors) that thecurrently-submitted fingerprint scan is non-fresh ornon-freshly-captured.

For example, the electronic device may detect, that a fingerprint scanwas allegedly captured and then was submitted for user authentication;however, during the 4 seconds (or K seconds) that preceded immediatelyprior to such submission, and/or during such submission operation,and/or during the 2 seconds (or N seconds) that immediately followedsuch submission, the device accelerometers have sensed no deviceacceleration, and/or have measured insufficient device acceleration(e.g., below a predefined threshold value) that does not correspond toan indication that the device was moved or tilted for capturing thealleged fresh fingerprint; and/or that the device gyroscope(s) havesensed no change in device orientation or position or tilt or slanting,and/or have measured insufficient change in device orientation orslanting or position or tilt (e.g., below a predefined threshold value)that does not correspond to an indication that the device was moved ortilted for capturing the alleged fresh fingerprint. In such situation,the device may determine that the alleged, freshly-submitted,fingerprint scan, was not accompanied (during the submission, and/orimmediately prior to the submission, and/or immediately after thesubmission) with geo-spatial changes of the device (or its acceleration,orientation, tilt, geo-spatial positioning or slanting) or withsufficient geo-spatial changes (e.g., based on threshold values, or arange of values), and therefore lack of “liveness” or lack of“freshness” is declared; and in turn, the user authentication is deniedor is rejected, or the user is required to perform additionalauthentication steps.

In some embodiments, a lookup table of reference values, and/or a rangeof reference values, as well as lookup operations or comparisonoperations of measured (or sensed) values relative to such referencevalues, may be used in order to determine sufficiency or insufficiencyof changes in device properties that accompany a touch-event or acapture (or submission) of a self-image or a capture (or submission) ofa fingerprint scan, or in order to establish whether or not the measuredor sensed spatial properties of the device match an “expected” set orrange of values that are “expected” to accompany a capturing or aninputting or a submission of such sample, or that are “expected” toprecede immediately prior to (or to follow immediately after) suchcapturing or submission or inputting of such sample.

For example, a lookup table or a pre-defined range of values, may beconfigured such that, device acceleration of at least K m/s² indicatessufficient device acceleration that accompanies a fresh capturing ofself-taken image or fingerprint scan or touch event or other biometricsample; whereas, device acceleration that is smaller than said value isinsufficient and the “fresh” sample is estimated to be illegitimate(e.g., an attacker, an impostor, a replay attack, or the like). Inanother example, a lookup table or a pre-defined range of values, may beconfigured such that, device orientation of at least K degrees relativeto a vertical plane, indicates sufficient device orientation thataccompanies a fresh capturing of self-taken image or fingerprint scan ortouch event or other biometric sample; whereas, device orientation thatis smaller than said value is insufficient and the “fresh” sample isestimated to be illegitimate (e.g., an attacker, an impostor, a replayattack, or the like). Other conditions may be used.

In some embodiments, the user authentication process, which is performedby a user authentication module or unit of the electronic device and/orby a user authentication module or unit of a remote server, furthertakes into account, a comparison or the similarity between: (i) thechanges in geo-spatial properties or characteristics of the deviceduring (and/or immediately before, and/or immediately after) the currentsubmission and/or the current alleged-capturing of fingerprint scan, and(ii) the changes in geo-spatial properties or characteristics of thedevice during (and/or immediately before, and/or immediately after) thetime in which the reference fingerprint scan was captured. The level ofsimilarity, e.g., optionally calculated by using a lookup table or asuitable weighted formula or equation, or by using reference values orpre-defined threshold values, may be used in order to confirm theidentity of the user and/or the authenticity of the submitted “fresh”fingerprint scan. For example, during (and/or immediately prior to,and/or immediately after) the capturing of the original, Reference,fingerprint scan, the electronic device measures and senses and storesthe geo-spatial properties of the device and the changes in suchproperties (e.g., device acceleration, orientation, tilt, slanting,spinning, rotation, lifting up, moving down, or the like); and it maygenerate and store a reference signature of one or more such changesthat accompanied the inputting of the Reference fingerprint scan. Forexample, when user Adam inputs his initial, Reference, fingerprint scan,user Adam accelerated the device at 30 degrees relative to the groundand/or has tilted the device 45 degrees horizontally immediately priorto scanning his finger (e.g., within the 2 seconds immediately prior tocapturing and/or submitting of the Reference fingerprint scan).Subsequently, attacker or impostor Bob attempts to authenticate via theelectronic device, and submits a “freshly captured” fingerprint scan (ora digital representation thereof); while doing so, attacker Bob movesthe electronic device, thereby causing the system to confirm Liveness;however, attacker Bob moves the electronic device in a different manner,during or before or after the submission or the capturing of the freshfingerprint scan, a manner that is different from the Reference Mannerthat the legitimate user Adam had exhibited when user Adam has capturedthe original, Reference, fingerprint scan (e.g., user Bob rotates thedevice counter-clockwise by 70 degrees prior to the scanning of thecurrent fingerprint; whereas user Adam did not exhibit such rotation,and instead had exhibited other geo-spatial changes to the device).

Accordingly, the electronic device (or a remote server) may compare andanalyze the properties of the electronic device, sensed immediatelybefore and/or during and/or immediately after the submission of thefresh fingerprint scan; and may compare the currently-sensed parametersto one or more of: (i) the historical parameters of the device(acceleration, device orientation, device tilt, device slanting, or thelike) that had accompanied, and were measured and sensed at, the initialcapturing of the Reference fingerprint scan; and/or (ii) a set of one ormore geo-spatial parameters of the device, that were captured during oneor more previous authentication sessions of that user; and/or (iii) anaverage, or weighted average, or other statistical formula or function,or previously-captured changes in the geo-spatial properties of theelectronic device during previous fingerprint-based authenticationsessions (e.g., detecting that Typically, use Adam rotates his deviceclockwise by 80 degrees, immediately prior to scanning his fingerprint;whereas the current user does not exhibit this change in properties ofthe device, that was sensed in 92 percent of the previous userauthentication sessions of user Adam); and/or (iv) the geo-spatialproperties of the device, as captured in a recent user-authenticationsession that was approved (e.g., within the past K days, the past 3days, the most-recent 7 days, or the like), or in the most-recentauthentication session that was approved.

The present invention may thus utilize the geo-spatial properties andcharacteristics of the electronic device, as well as changes in suchgeo-spatial properties and characteristics, that are sensed and measuredby the device during and/or immediately prior to and/or immediatelysubsequent to the submission (and/or the capturing) of the “current”fingerprint scan, in order to: (i) confirm or detect “liveness” or“freshness” of the currently-submitted fingerprint scan; and/or (ii)reject a fingerprint scan that is estimated to be non-fresh or that isestimate to lack liveness or to lack sufficient liveness properties;and/or (iii) compare the currently-measured geo-spatial properties ofthe device, to previously-sensed geo-spatial properties that hadaccompanied the inputting of the Reference fingerprint scan, in order todetect a possible attacker or impostor; and/or (iv) compare thecurrently-measured geo-spatial properties of the device, topreviously-sensed geo-spatial properties that had accompanied theinputting of one or more previous fingerprint scan(s), in order todetect a possible attacker or impostor.

It is clarified that the system or the device may take into account,analysis of sensed device parameters (e.g., geo-spatial parameters,acceleration, deceleration, device orientation, device tilt, deviceslanting, or the like) that “accompany” the submission and/or thecapturing of a user-authentication factor (e.g., a fingerprint scan; aretina scan; entry of a password or PIN or pass-phrase; entry of abiometric sample, voice sample, speech sample, audio sample; capturingof a “selfie” or “selfy” image or photograph or video-clip; auser-specific behavioral parameter that can be extracted from trackingof user interactions with the electronic device and/or with thecomputerized system; a user-specific cognitive parameter that can beextracted from tracking of user interactions with the electronic deviceand/or with the computerized system; a user-specific physiologicalparameter that can be extracted from tracking of user interactions withthe electronic device and/or with the computerized system; a level or anintensity (or other characteristic) of force or pressure that the userapplies to a touch-screen or to a touch-pad or to other touch-based unitor input unit or interface; or the like); wherein such “accompanying”parameters are sensed or captured during the submission of theuser-authentication factor and/or immediately prior to such submissionand/or immediately following such submission; and/or wherein such“accompanying” parameters are sensed and captured during the alleged orthe proffered capturing of the currently-submitted fingerprint orimmediately prior to such alleged capturing or immediately followingsuch alleged capturing. It is further clarified that although portionsof the discussion herein my relate, for demonstrative purposes, to aself-taken image and/or to a fingerprint scan as biometric factors or asuser-authentication factors, the present invention may be utilized inconjunction with other biometric features and/or other biometric samplesand/or other user-authentication factors, whose liveness and/orfreshness and/or authenticity may thus be confirmed or detected (ordenied) by using the mechanisms of the present invention.

The similarity, or the level of similarity, between the currently-sensedgeo-spatial properties or device properties, that are sensed asaccompanying a current submission or a current alleged capturing of afresh sample (e.g., self-image, fingerprint scan), may be compared to aprevious set of reference values, or to a set of one or more thresholdvalues. Comparison operations may be used to determine whethersufficient similarity is detected. For example, if the reference samplewas captured when the device exhibits a rotation counter-clockwise of 85degrees; and a fresh sample is captured when the device exhibits arotation counter-clockwise of 79 degrees; then, the device or thecomputerized system may determine that the two properties aresufficiently close to each other, since, for example, thefreshly-measured value (counter-clockwise rotation) is the same type ofdevice property as the reference measurement (counter-clockwiserotation), and the currently-measured numerical value (79 degrees ofrotation) is within K percent of the reference measurement value (85degrees of rotation), such as, K may be configured as 5 percent or 10percent of 16 percent or other suitable similarity level that allowssome pre-defined level of difference while still enabling to detectsimilarity. Additionally or alternatively, a Range of values may be usedto determine similarity or sufficient similarity; such as, for example,the device may be configured to accept as “sufficiently similar” acontemporary counter-clockwise rotation in the range of 75 to 95 degrees(namely, plus or minus ten degrees relative to the Reference value).Other suitable methods may be used to determine sufficient similarity ormatch or matching of current and historical (or reference) geo-spatialproperties or device properties.

Some embodiments may include a method comprising: (a) generating auser-authentication process that utilizes at least a self-taken image ofa user of an electronic device, as an authentication factor forauthenticating said user to a computerized service; (b) in saiduser-authentication process, determining whether: (I) a submission bysaid user of said self-taken image, follows after a time-period in whichgeo-spatial characteristics of the electronic device indicate that theelectronic device was utilized for capturing said self-image of theuser, or (II) the submission by said user of said self-taken image, doesnot follow a time-period in which geo-spatial characteristics indicatethat the electronic device was utilized for capturing said self-image ofthe user; (c) if it is determined that the submission by said user ofsaid self-taken image, does not follow a time-period in whichgeo-spatial characteristics indicate that the electronic device wasutilized for capturing said self-image of the user, then: triggering analert that said self-taken image is estimated to be anot-freshly-captured image of said user.

In some embodiments, the method comprises: sensing via an accelerometerof the electronic device, device-acceleration data that accompanies thesubmission by said user of said self-taken image; determining thatdevice-acceleration data, that accompanies the submission by said userof said self-taken image, does not correspond to device-accelerationdata that is expected to accompany submission of a fresh self-takenimage; and based on said determining, triggering a determination thatsaid self-taken image is estimated to be a not-freshly-captured image ofsaid user.

In some embodiments, the method comprises: sensing via a gyroscope ofthe device, device-orientation data that accompanies the submission bysaid user of said self-taken image; determining that device-orientationdata, that accompanies the submission by said user of said self-takenimage, does not correspond to device-orientation data that is expectedto accompany submission of a fresh self-taken image; and based on saiddetermining, triggering a determination that said self-taken image isestimated to be a not-freshly-captured image of said user.

In some embodiments, the method comprises: sensing via an accelerometerof the electronic device, device-acceleration data immediately prior tothe submission by said user of said self-taken image; determining thatdevice-acceleration data, that was sensed immediately prior to thesubmission by said user of said self-taken image, does not correspond todevice-acceleration data that is expected to precede submission of afresh self-taken image; and based on said determining, triggering adetermination that said self-taken image is estimated to be anot-freshly-captured image of said user.

In some embodiments, the method comprises: sensing via a gyroscope ofthe electronic device, device-orientation data immediately prior to thesubmission by said user of said self-taken image; determining thatdevice-orientation data, that was sensed immediately prior to thesubmission by said user of said self-taken image, does not correspond todevice-orientation data that is expected to precede submission of afresh self-taken image; and based on said determining, triggering adetermination that said self-taken image is estimated to be anot-freshly-captured image of said user.

In some embodiments, a process comprises: (a) generating auser-authentication process that utilizes at least a self-taken image ofa user of an electronic device, as an authentication factor forauthenticating said user to a computerized service; (b) capturing areference self-image of the user, to be used subsequently as anauthentication factor for authenticating said user; (c) during saidcapturing of the reference self-image of the user, sensing also spatialproperties of said electronic device, via at least one of: anaccelerometer of said electronic device, and a gyroscope of saidelectronic device; (d) subsequently, receiving from said user acurrently-submitted self-image, that the user submits as auser-authentication factor; (e) determining whether or not during atime-period, that is immediately prior to submission of thecurrently-submitted self-image, the electronic device sensed spatialdevice properties that are at least similar, beyond a pre-definedthreshold of similarity, to the spatial device properties that weresensed during the capturing of the reference self-image in step (c); and(f) if it is determined that during said time-period, that isimmediately prior to submission of the currently-submitted self-image,the electronic device sensed spatial device properties that are at leastsimilar, beyond a pre-defined threshold of similarity, to the spatialdevice properties data that was sensed during the capturing of thereference self-image in step (c), then: triggering an alert that saidself-taken image is estimated to be a not-freshly-captured image of saiduser.

In some embodiments, step (c) comprises: during said capturing of thereference self-image of the user, sensing device-acceleration data ofsaid electronic device, via the accelerometer of said electronic device;wherein step (e) comprises: determining whether or not during thetime-period, that is immediately prior to submission of thecurrently-submitted self-image, the accelerometer of the electronicdevice sensed device-acceleration data that is least similar, beyond apre-defined threshold of similarity, to the device-acceleration datathat was sensed during the capturing of the reference self-image in step(c); wherein step (f) comprises: if it is determined that during thetime-period, that is immediately prior to submission of thecurrently-submitted self-image, the accelerometer of the device did notsense device-acceleration data that is least similar, beyond apre-defined threshold of similarity, to the device-acceleration datathat was sensed during the capturing of the reference self-image in step(c), then: triggering an alert that said self-taken image is estimatedto be a not-freshly-captured image of said user.

In some embodiments, step (c) comprises: during said capturing of thereference self-image of the user, sensing device-orientation data ofsaid electronic device, via the gyroscope of said electronic device;step (e) comprises: determining whether or not during the time-period,that is immediately prior to submission of the currently-submittedself-image, the gyroscope of the electronic device senseddevice-orientation data that is least similar, beyond a pre-definedthreshold of similarity, to the device-orientation data that was sensedduring the capturing of the reference self-image in step (c); step (f)comprises: if it is determined that during the time-period, that isimmediately prior to submission of the currently-submitted self-image,the gyroscope of the electronic device did not sense device-orientationdata that is least similar, beyond a pre-defined threshold ofsimilarity, to the device-orientation data that was sensed during thecapturing of the reference self-image in step (c), then: triggering analert that said self-taken image is estimated to be anot-freshly-captured image of said user.

In some embodiments, a method comprises: (a) generating auser-authentication process that utilizes at least a fingerprint scan ofa user of an electronic device, captured via a fingerprint scanner ofsaid electronic device, as an authentication factor for authenticatingsaid user to a computerized service; (b) in said user-authenticationprocess, determining whether (I) a submission by said user of saidfingerprint scan, is accompanied by sensed spatial characteristics ofthe electronic device which indicate that the electronic device is beingutilized for capturing said fingerprint scan, or (II) the submission bysaid user of said fingerprint scan, is accompanied by sensed spatialcharacteristics of the electronic device which indicate that theelectronic device is not being utilized for capturing said fingerprintscan; (c) if it is determined that the submission by said user of saidfingerprint scan, is accompanied by sensed spatial characteristics ofthe electronic device which indicate that the electronic device is notbeing utilized for capturing said fingerprint scan, then: triggering analert that said fingerprint scan is estimated to be anot-freshly-captured fingerprint scan.

In some embodiments, the method comprises: sensing via an accelerometerof the electronic device, device-acceleration data that accompanies thesubmission by said user of said fingerprint scan; determining thatdevice-acceleration data, that accompanies the submission by said userof said fingerprint scan, does not correspond to device-accelerationdata that is expected to accompany submission of a fresh fingerprintscan; and based on said determining, triggering a determination thatsaid fingerprint scan is estimated to be a not-freshly-captured image ofsaid user.

In some embodiments, the method comprises: sensing via a gyroscope ofthe electronic device, device-orientation data that accompanies thesubmission by said user of said fingerprint scan; determining thatdevice-orientation data, that accompanies the submission by said user ofsaid fingerprint scan, does not correspond to device-orientation datathat is expected to accompany submission of a fresh fingerprint scan;and based on said determining, triggering a determination that saidfingerprint scan is estimated to be a not-freshly-captured image of saiduser.

In some embodiments, the method comprises: sensing via an accelerometerof the electronic device, device-acceleration data immediately prior tothe submission by said user of said fingerprint scan; determining thatdevice-acceleration data, that was sensed immediately prior to thesubmission by said user of said fingerprint scan, does not correspond todevice-acceleration data that is expected to precede submission of afresh fingerprint scan; and based on said determining, triggering adetermination that said fingerprint scan is estimated to be anot-freshly-captured fingerprint scan of said user.

In some embodiments, the method comprises: sensing via a gyroscope ofthe electronic device, device-orientation data immediately prior to thesubmission by said user of said fingerprint scan; determining thatdevice-orientation data, that was sensed immediately prior to thesubmission by said user of said fingerprint scan, does not correspond todevice-orientation data that is expected to precede submission of afresh fingerprint scan; and based on said determining, triggering adetermination that said fingerprint scan is estimated to be anot-freshly-captured fingerprint scan of said user.

In some embodiments, a process comprises: (a) generating auser-authentication process that utilizes at least a fingerprint scan ofa user of an electronic device, captured via a fingerprint scanner ofsaid electronic device, as an authentication factor for authenticatingsaid user to a computerized service; (b) capturing a referencefingerprint scan of the user, to be used subsequently as anauthentication factor for authenticating said user; (c) during saidcapturing of the reference fingerprint scan, sensing also accelerationdata/gyroscope data/device-orientation data of said electronic device,from an accelerometer/gyroscope of said electronic device; (d)subsequently, receiving from said user a currently-submitted fingerprintscan, that the user submits as a user-authentication factor; (e)determining whether or not during a time-period, that is immediatelyprior to submission of the currently-submitted fingerprint scan, theelectronic device sensed acceleration data/gyroscopedata/device-orientation data that are at least similar, beyond apre-defined threshold of similarity, to the acceleration data/gyroscopedata/device-orientation data that was sensed during the capturing of thereference fingerprint scan in step (c); (f) if it is determined thatduring a time-period, that is immediately prior to submission of thecurrently-submitted fingerprint scan, the electronic device did notacceleration data/gyroscope data/device-orientation data that are atleast similar, beyond a pre-defined threshold of similarity, to theacceleration data/gyroscope data/device-orientation data that was sensedduring the capturing of the reference fingerprint scan in step (c),then: triggering an alert that the currently-submitted fingerprint scanis estimated to be a not-freshly-captured fingerprint scan.

In some embodiments, step (c) comprises: during said capturing of thereference fingerprint scan of the user, sensing device-acceleration dataof said electronic device, via the accelerometer of said electronicdevice; wherein step (e) comprises: determining whether or not duringthe time-period, that is immediately prior to submission of thecurrently-submitted fingerprint scan, the accelerometer of theelectronic device sensed device-acceleration data that is least similar,beyond a pre-defined threshold of similarity, to the device-accelerationdata that was sensed during the capturing of the reference fingerprintscan in step (c); wherein step (f) comprises: if it is determined thatduring the time-period, that is immediately prior to submission of thecurrently-submitted fingerprint scan, the accelerometer of theelectronic device did not sense device-acceleration data that is leastsimilar, beyond a pre-defined threshold of similarity, to thedevice-acceleration data that was sensed during the capturing of thereference fingerprint scan in step (c), then: triggering an alert thatsaid self-taken image is estimated to be a not-freshly-capturedfingerprint scan of said user.

In some embodiments, step (c) comprises: during said capturing of thereference fingerprint scan of the user, sensing device-orientation dataof said electronic device, via the gyroscope of said electronic device;wherein step (e) comprises: determining whether or not during thetime-period, that is immediately prior to submission of thecurrently-submitted fingerprint scan, the gyroscope of the electronicdevice sensed device-orientation data that is least similar, beyond apre-defined threshold of similarity, to the device-orientation data thatwas sensed during the capturing of the reference fingerprint scan instep (c); wherein step (f) comprises: if it is determined that duringthe time-period, that is immediately prior to submission of thecurrently-submitted fingerprint scan, the gyroscope of the electronicdevice did not sense device-orientation data that is least similar,beyond a pre-defined threshold of similarity, to the device-orientationdata that was sensed during the capturing of the reference fingerprintscan in step (c), then: triggering an alert that said fingerprint scanis estimated to be a not-freshly-captured fingerprint scan of said user.

The terms “self-taken item” or “self-taken visual item” or “self-takencamera-acquired item” or “self-taken visual acquisition item” or“self-taken visual acquisition input”, as used herein, may include aself-taken image (“selfie” image) and/or a self-taken photograph and ora self-taken video (“selfie” video) and/or a self-taken batch-of-imagesand/or a self-taken group of video frames, or other visual orcamera-based input that may be utilized for signing-in or logging-in orface-based sign in or user authentication.

In accordance with the present invention, a user authentication unit ormodule or device may capture (via one or more cameras) a selfie image ora selfie video, or a set or series of selfie images or frames, which arethen utilized for authenticating the user; and also for determiningwhether the submitted or captured image or video is a “fresh” one or afreshly-captured one or a recently-captured one, rather than being apreviously-captured or previously-submitted item (e.g., captured orsubmitted a minute or an hour or a day or a year prior to the currentauthentication attempt); and also for determining Liveness, namely, thatthe captured image or video reflect therein a live human user, ratherthan (for example) being an image (or a video) of a two-dimensionalprinted photograph of a person.

In some embodiments, for example, the user authentication unit maycomprise a Video/Image Content Analyzer, which may perform computervision analysis on video or video frames or image(s) captured orsubmitted for authentication purposes, in order to extract from suchimaged or captured or submitted content, one or more features that maybe correlated or conversely may be contradicted with data that is sensedor measured or captured by one or more other sensors or units of theelectronic device that allegedly (or factually) submitted or capturedsuch video or image(s); such as, features or data acquired by one ormore accelerometer(s), gyroscope unit(s), compass unit(s), deviceorientation sensor(s), device spatial-orientation sensor(s), deviceslanting/tilt sensor, device location-determining sensor, GlobalPositioning System (GPS) sensor, or the like. If a mismatch isdetermined or is found, between (i) content that appears in thesubmitted video or image, and (ii) data that is collected or sensed byor from such sensors of the submitting electronic device, then, the userauthentication unit may deny or unauthorize the access, or may reject ordeny the authentication attempt or the log-in attempt, or may respondnegatively to such authentication attempt or log-in attempt, or maydeclare that the user is not authenticated; or in some embodiments, maytrigger or may initiate one or more fraud mitigation operations, forexample, requiring the user to perform authentication via an additionalfactor (e.g., to answer pre-defined security questions) or to perform atwo-factor authentication (TFA) or multi-factor authentication (MFA) orto contact a customer service representative by phone or in person, orthe like.

In a first set of examples, the user authentication unit, which may belocated within the electronic device and/or may be part of a remoteserver, receives a submitted selfie image or selfie video. The contentanalyzer unit performs computer vision analysis of the content shown,and detects that the submitted three-seconds video (or, that thesubmitted 2 or 8 or 60 consecutive frames that were taken during theselfie acquisition) shows the user and/or its background in anabsolutely still and idle position, such that the submitted video (orgroup of images) does not reflect any type of shaking or movement of thehand of the user that allegedly took the video or images. For example,content analysis of the submitted video, or of the submitted set ofimages, shows the user's head being located at exactly the same positionwithin the frame(s) or video or image(s); for example, an image of a“head” appears in the center of the frame, and does not have any offsetor any displacement from one frame to another frame. This is highlyunusual or irregular in a true acquisition of a three-second selfievideo; as even the most proficient human user would typically cause someshaking or vibration or movement or instability or steadiness of theelectronic device during the acquisition period. Furthermore, the userauthentication unit may now correlate or contradict or match thiscontent from the selfie video or image(s), with data collected by thesensors of the electronic device. For example, data sensed or measuredby the accelerometer(s) and/or gyroscope unit(s) and/or compass unit(s)and/or device orientation sensor(s) and/or device spatial-orientationsensor(s) and/or device slanting/tilt sensor(s) and/or devicelocation-determining sensor(s) and/or device's GPS sensor, as capturedduring the capturing or the acquisition or the submitting of the selfieimage or selfie video, or within T seconds before and/or after suchacquisition or submission (e.g., T being 1 or 2.5 or 3 seconds),indicate that such sensor(s) do report movement or instability ornon-steadiness or motion or acceleration or deceleration or shaking ofthe electronic device. Accordingly, the system determines that thestable and idle or steady (non-moving) Content that is recognized in thesubmitted selfie video or selfie image(s), contradicts or does not matchor does not correlate with the data or features as extracted or learnedfrom the device's sensor(s). Such determination by the system may thuslead to denial of the authentication request or rejection of theauthentication attempt.

In a second set of examples, the user authentication unit, which may belocated within the electronic device and/or may be part of a remoteserver, receives a submitted selfie image or selfie video. The contentanalyzer unit performs computer vision analysis of the content shown,and detects that the submitted four-seconds video (or, that thesubmitted 4 or 9 or 50 consecutive frames that were taken during theselfie acquisition) shows the user and/or its background in a mannerthat indicates that the capturing device was slightly shaking or movingor was instable or non-steady or non-idle, such that the submitted video(or group of images) reflects at least some shaking or movement of thehand of the user that allegedly took the video or images. For example,content analysis of the submitted video, or of the submitted set ofimages, shows the user's head being located at slightly differentpositions within the frame(s) or video or image(s); for example, animage of a “head” appears in the center frame #1, and then appears at anoffset of one pixel to the right of the center in frame #2, and thenappears at an offset of three pixels to the left of the center in frame#3, and so forth, reflecting some displacement from one frame to anotherframe. This is usual or regular in a true acquisition of a four-secondselfie video; as even the most proficient human user would typicallycause some shaking or vibration or movement or instability ornon-steadiness of the electronic device during the acquisition period.However, the user authentication unit may now correlate or contradict ormatch this content from the selfie video or image(s), with datacollected by the sensors of the electronic device. For example, datasensed or measured by the accelerometer(s) and/or gyroscope unit(s)and/or compass unit(s) and/or device orientation sensor(s) and/or devicespatial-orientation sensor(s) and/or device slanting/tilt sensor(s)and/or device location-determining sensor(s) and/or device's GPS sensor,as captured during the capturing or the acquisition or the submitting ofthe selfie image or selfie video, or within T seconds before and/orafter such acquisition or submission (e.g., T being 1 or 2.5 or 3seconds), indicate that such sensor(s) do not report any movement orinstability or non-steadiness or motion or acceleration or decelerationor shaking of the electronic device. Accordingly, the system determinesthat the shaking or non-stable or non-steady or non-idle Content that isrecognized in the submitted selfie video or selfie image(s), contradictsor does not match or does not correlate with the data or features asextracted or learned from the device's sensor(s) which indicate an idledevice or a non-moving device or a completely stable device. Suchdetermination by the system may thus lead to denial of theauthentication request or rejection of the authentication attempt.

In a third set of examples, the user authentication unit, which may belocated within the electronic device and/or may be part of a remoteserver, receives a submitted selfie image or selfie video. The contentanalyzer unit performs computer vision analysis of the content shown,and detects that the submitted four-seconds video (or, that thesubmitted 4 or 9 or 50 consecutive frames that were taken during theselfie acquisition) shows the user and/or its background in a mannerthat indicates that the capturing device was slightly shakinghorizontally or moving horizontally or was horizontally instable or washorizontally non-idle or was horizontally non-steady, such that thesubmitted video (or group of images) reflects horizontal shaking ormovement of the hand of the user that allegedly took the video orimages; while also reflecting (as determined by the content analyzer)non-shaking or non-movement or steadiness in the vertical direction orthe vertical axis. For example, content analysis of the submitted video,or of the submitted set of images, shows the user's head being locatedat slightly different positions within the frame(s) or video orimage(s); for example, an image of a “head” appears in the center frame#1, and then appears at an offset of one pixel to the right of thecenter in frame #2, and then appears at an offset of three pixels to theleft of the center in frame #3, and so forth, reflecting some Horizontaldisplacement from one frame to another frame. This may be usual orregular in a true acquisition of a four-second selfie video; as even themost proficient human user would typically cause some shaking orvibration or movement or instability or non-steadiness of the electronicdevice during the acquisition period. However, the user authenticationunit may now correlate or contradict or match this content from theselfie video or image(s), with data collected by the sensors of theelectronic device. For example, data sensed or measured by theaccelerometer(s) and/or gyroscope unit(s) and/or compass unit(s) and/ordevice orientation sensor(s) and/or device spatial-orientation sensor(s)and/or device slanting/tilt sensor(s) and/or device location-determiningsensor(s) and/or device's GPS sensor, as captured during the capturingor the acquisition or the submitting of the selfie image or selfievideo, or within T seconds before and/or after such acquisition orsubmission (e.g., T being 1 or 2.5 or 3 seconds), indicate that suchsensor(s) do indeed report movement or instability or motion oracceleration or deceleration or shaking of the electronic device, butsuch movement or instability or shaking are actually reported by thedevice's sensor(s) as being Vertical and not horizontal, or as beingDiagonal (e.g., both vertical and horizontal; and not only horizontal).Accordingly, the system determines that the Horizontal shaking ornon-stable or non-idle Content that is recognized in the submittedselfie video or selfie image(s), contradicts or does not match or doesnot correlate with the data or features as extracted or learned from thedevice's sensor(s) which indicate a Vertically moving device, or avertically-and-horizontally moving device. Such determination by thesystem may thus lead to denial of the authentication request orrejection of the authentication attempt.

In a fourth set of examples, the user authentication unit, which may belocated within the electronic device and/or may be part of a remoteserver, receives a submitted selfie image or selfie video. The contentanalyzer unit performs computer vision analysis of the content shown,and detects that the submitted four-seconds video (or, that thesubmitted 4 or 9 or 50 consecutive frames that were taken during theselfie acquisition) shows the user and/or its background in a mannerthat indicates that the capturing device was only Slightly shakinghorizontally or only Slightly moving horizontally or was horizontallyslightly instable or was horizontally slightly non-idle, such that thesubmitted video (or group of images) reflects a small amount ofhorizontal shaking or movement of the hand of the user that allegedlytook the video or images; while also reflecting (as determined by thecontent analyzer) non-shaking or non-movement in the vertical directionor the vertical axis. For example, content analysis of the submittedvideo, or of the submitted set of images, shows the user's head beinglocated at slightly different positions within the frame(s) or video orimage(s); for example, an image of a “head” appears in the center frame#1, and then appears at an offset of one pixel to the right of thecenter in frame #2, and then appears at an offset of three pixels to theleft of the center in frame #3, and so forth, reflecting some Horizontaldisplacement from one frame to another frame. This may be usual orregular in a true acquisition of a four-second selfie video; as even themost proficient human user would typically cause some shaking orvibration or movement or instability of the electronic device during theacquisition period. However, the user authentication unit may nowcorrelate or contradict or match this content from the selfie video orimage(s), with data collected by the sensors of the electronic device.For example, data sensed or measured by the accelerometer(s) and/orgyroscope unit(s) and/or compass unit(s) and/or device orientationsensor(s) and/or device spatial-orientation sensor(s) and/or deviceslanting/tilt sensor(s) and/or device location-determining sensor(s)and/or device's GPS sensor, as captured during the capturing or theacquisition or the submitting of the selfie image or selfie video, orwithin T seconds before and/or after such acquisition or submission(e.g., T being 1 or 2.5 or 3 seconds), indicate that such sensor(s) haveactually reported gross or significant or non-slight movement orinstability or motion or acceleration or deceleration or shaking of theelectronic device, for example, indicating that the end-user device wasactually rotated or spun horizontally in a manner that corresponds to 45or 60 degrees of the 360 degrees around it; and therefore, while theContent of the selfie video (or group of images) show the user's head asmoving or displaced by just a few pixels to the right or to the left, aTrue selfie video (or set of images) would have shown a largedisplacement of the user's head from frame to frame by at least 30 or 50pixels, or would have even shown the left half of the user's headbecoming outside of the field-of-view being acquired. Accordingly, thesystem determines that the Slight or negligible horizontal shakingreflected in the Content that is recognized in the submitted selfievideo or selfie image(s), contradicts or does not match or does notcorrelate with the data or features as extracted or learned from thedevice's sensor(s) which indicate a Gross or non-negligible movement orshaking. Such determination by the system may thus lead to denial of theauthentication request or rejection of the authentication attempt.

In a fifth set of examples, the user authentication unit, which may belocated within the electronic device and/or may be part of a remoteserver, receives a submitted selfie image or selfie video. The contentanalyzer unit performs computer vision analysis of the content shown,and detects that the submitted four-seconds video (or, that thesubmitted 4 or 9 or 50 consecutive frames that were taken during theselfie acquisition) shows the user's head located clearly on a pillowwithin a bed, corresponding to a user that is lying down in a bed andholding his smartphone above his head with the smartphone beinggenerally parallel to the ground and facing downwardly towards thepillow and the user's head. However, the user authentication unit maynow correlate or contradict or match this content from the selfie videoor image(s), with data collected by the sensors of the electronicdevice. For example, data sensed or measured by the accelerometer(s)and/or gyroscope unit(s) and/or compass unit(s) and/or deviceorientation sensor(s) and/or device spatial-orientation sensor(s) and/ordevice slanting/tilt sensor(s) and/or device location-determiningsensor(s) and/or device's GPS sensor, as captured during the capturingor the acquisition or the submitting of the selfie image or selfievideo, or within T seconds before and/or after such acquisition orsubmission (e.g., T being 1 or 2.5 or 3 seconds), indicate that suchsensor(s) have actually reported that the smartphone was continuouslybeing held Vertically and perpendicular to the ground, and notHorizontally; and/or that the smartphone was held horizontally but withits screen and its front-side camera Facing Upwardly towards theceiling. Accordingly, the system determines that the location ofpositioning of the user or his head or body or body-organ(s) asrecognized in the Video content or the Image(s) content, and/or theSurrounding of the user (e.g., pillow, bed, floor, walls), do not matchor do not correlate with, or even contradict, the spatial orientation orthe spatial positioning of the end-user device during (and/orimmediately before, and/or immediately after) the acquisition or thesubmission of the selfie video or selfie image(s). In another example,the computer vision analysis or the image/video recognition process, mayindicate that the user is standing against a wall and extending his armforward (e.g., as common in taking a selfie photo) such that thesmartphone is generally perpendicular to the ground; whereas, the actualdata sensed by the end-user device, during or before or after submissionor acquisition, indicates that the end-user device was slanted at 45degrees relative to the ground, such that it would have been impossiblefor a user to capture such photo from that angle or device slanting ordevice orientation. Such determination(s) by the system may thus lead todenial of the authentication request or rejection of the authenticationattempt.

In another set of examples, the content analysis of the selfie video orthe selfie image(s), indicates that they show the user being clearlylocated indoors, as the content analyzer recognizes image-portions thatcorrespond to indoor items (e.g., a floor lamp, a television, amicrowave oven in the background, a framed painting hanging on thewall); however, the user authentication unit checks and finds that thegeographical location of the end-user device, as determined or sensedand reported by its GPS sensor or other location-determining unit (e.g.,based on Wi-Fi based location determination; based on Cellular basedlocation determination or triangulation; or the like) clearly indicatethat the end-user device is actually located in Central Park in New YorkCity, or is located in the Grand Canyon in Nevada, or in an area orvenue that is clearly an outdoors area or a non-indoors area. Suchcontradiction or mismatch, between (i) the content shown in the selfievideo or the selfie image(s), and (ii) the content or surrounding orambient items that are expected to be (or, that are expected not to be)in the geographical location of the end-user device as reported by itslocation-determining unit, may lead the user authentication unit to denyor reject the authentication request.

Similarly, in another example, content analysis of the selfie video orselfie image shows the user standing in a busy street with many vehiclespassing by or passing behind him; whereas, GPS or location-based data ofthe end-user device, indicates that the end-user device is located atthe Grand Canyon in Nevada, or is located inside a particular SportsStadium, or inside a geographical region that is known to bevehicle-free (e.g., a forest, or a nature center); and such detectedcontradiction may lead to denial or rejection of the authenticationrequest.

In another example, the user authentication unit determines that the GPSsensor and/or the Wi-Fi transceiver of the end-user device, indicateclearly that the end-user device is located in Miami; whereas theContent of the selfie video or selfie image, clearly show the userwearing a fur coat and being located in a back yard that is visiblycovered or blanketed in heavy snow. The system may determine, based onpre-defined rules, that if Snow is detected in the video or image, butin contrast the location-determining sensor(s) indicate that theend-user device is in a snow-less region (or in a reason that does nothave snow at this time of the year), then the user authenticationrequest would be denied or rejected.

In another example, the user authentication unit determines that the GPSsensor and/or the Wi-Fi transceiver of the end-user device, indicateclearly that the end-user device is located at a location that,according to current weather data (e.g., obtained from a weatherreporting server, or from the Internet) currently has sunny weather withno snow and no rain; whereas, the selfie image or selfie video that wasjust submitted, allegedly from that location, clearly shows a userholding an umbrella under a cloudy sky with heavy rain. Such mismatchmay cause the system to reject or deny the authentication request, or atleast to require additional authentication factor(s) to be utilized.

In another set of examples, the content analysis of the selfie image(s)or selfie video(s), may even perform more advanced analysis which mayoptionally comprise Optical Character Recognition (OCR). For example,the content analysis of the selfie video or image(s), shows the userstanding at a street intersection, next to a street sign that says“Broadway” and “West 57th Street”; or next to a sign that says “MadisonSquare Garden”, or next to a sign that says “Carnegie Hall”; therebyenabling the content analysis unit to determine that the locationdepicted in the selfie item is a particular intersection or location inManhattan, N.Y. However, the user authentication unit checks and findsthat the GPS unit or other location-determining unit of the end-userdevice, indicates that the location of the device is actually LosAngeles, Calif. Such mismatch or contradiction may cause the userauthentication unit to reject or deny the authentication attempt.

In another example, the content analysis of the selfie image(s) orselfie video(s), shows the user standing at a street intersection, nextto a street sign that says “Avenue de Champs-Elysees” and “Rue Honore deBalzac”, and optionally also showing street signs in French language;thereby enabling the content analysis unit to determine that thelocation depicted in the selfie item is in Paris, France. However, theuser authentication unit checks and finds that the GPS unit or otherlocation-determining unit of the end-user device, indicates that thelocation of the device is actually Austin, Tex. Such mismatch orcontradiction may cause the user authentication unit to reject or denythe authentication attempt.

In another set of examples, the content analysis of the selfie image(s)or selfie video, shows the user standing outdoors (e.g., showing streetenvironment and vehicles passing behind him), and further showing sunnyblue sky above him or over his shoulders or over his head; therebyindicating to the user authentication unit that the selfie image orselfie video was captured at daytime. However, the GPS component orother location-determining unit of the electronic device, reports thatthe electronic device is currently located in California; and the userauthentication system checks what is the current local time inCalifornia and finds that it is currently 1:30 AM. The userauthentication system thus detects a contradiction or mismatch ornon-correlation, between (i) the time-of-day as deduced from computervision analysis of the selfie image or selfie video, and (ii) thetime-of-day as deduced from the geographic location that is reported bythe GPS component or other location-determining component of theelectronic device. Such mismatch or contradiction may cause the userauthentication unit to reject or deny the authentication attempt. In anopposite example, content analysis of the selfie image or selfie videoindicate that they were taken at night-time (e.g., showing therein theuser standing outdoor, with dark black sky over his head, and with litstreet lamps and lit houses behind him); yet the GPS component (or otherlocation-determining sensor) of the electronic device has sensed andreported that the electronic device is actually located in Florida, inwhich the current time now is 11 AM which corresponds to day-time andnot night-time; and Such mismatch or contradiction may cause the userauthentication unit to reject or deny the authentication attempt.

All the above are various non-limiting examples of user authenticationprocesses and determinations and comparisons that may be performed bysome embodiments of the present invention. Other suitable determinationsor rules or lookup tables may be used, to determine a mismatch ornon-correlation or even a clear contradiction, between (i) one or morefeatures that are extracted or derived or deduced from Content Analysisof a selfie video or selfie image(s), and (ii) data sensed or measuredor captured by the end-user device sensors (accelerometer, compass,gyroscope, device-orientation sensor, GPS, location-determining sensor,or the like). In some embodiments, computer vision algorithms may beused, to extract or to deduce from the selfie image(s) or selfie video,indications of the end-user device's position, orientation, spatialorientation, slanting, tilting, manner of being held by the user (e.g.,horizontally, vertically, diagonally, upside down, facing down, facingup, facing sideway), being idle or stable or being moving or non-stableor non-idle or shaking, the angle or slanting of the device relative tothe floor or ground or ceiling, the geo-spatial or geographic locationof the device, indoor indications, outdoor indications, weather orweather-related indications, or the like; and such indications may becompared to or matched with data sensed or measured or reported by therelevant sensors of the end-user device. A contradiction or mismatch maythus lead to rejection or denial of the authentication attempt, or atleast to requiring the user to utilize an additional authenticationfactor or authentication means.

One or more computer vision algorithms or image recognition algorithms,may be used for analysis of the content of the selfie image(s) or selfievideo; for example, image recognition based on prior training with alarge data-set of items or objects, or based on machine learning (ML) ordeep learning or artificial intelligence (AI), or by using a NeuralNetwork (NN), or based on object recognition, computer vision algorithmsbased on edge matching and/or edge detection, or based on CAD-likeobject models, or using appearance-based object recognition methods(e.g., based on comparison with known example images or known exemplarsor known templates), divide-and-conquer image searches that divide animage or a frame into cells or blocks, grayscale matching algorithms(e.g., computing pixel distance as a function of both pixel position andpixel intensity), gradient matching, methods that utilize eigenvectorsof templates (or eigenfaces), feature-based recognition methods, methodsthat utilize image interpretation tree(s), pose consistency, poseclustering, geometric hashing, scale-invariant feature transform (SIFT),speeded up robust features (SURF), or the like.

In some embodiments, a method comprises: (a) generating auser-authentication process for authenticating a user of an electronicdevice, wherein the user-authentication process utilizes a self-takenvisual-acquisition input of said user, wherein the self-takenvisual-acquisition input comprises one of: a self-taken image, aself-taken video; (b) performing image recognition on content shown insaid self-taken visual-acquisition input, and extracting a particularfeature based on image recognition of said content; (c) determining acharacteristic of said electronic device, based on data sensed by one ormore non-camera sensors of said electronic device during acquisition ofsaid self-taken visual-acquisition input; (d) detecting a mismatchbetween (I) said particular feature that was extracted in step (b) basedon image recognition of the content of the self-taken visual-acquisitioninput, and (II) said characteristic of the electronic device that wasdetermined in step (c) based on data sensed by said one or morenon-camera sensors of said electronic device; (e) based on said mismatchdetected in step (d), denying an authentication request received fromsaid electronic device.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was held vertically during acquisition; detecting that aspatial-orientation sensor of the electronic device, indicates that theelectronic device was held non-vertically during acquisition; based onthe above detecting operations, determining said mismatch, and denyingsaid authentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was held horizontally during acquisition; detecting that aspatial-orientation sensor of the electronic device, indicates that theelectronic device was held non-horizontally during acquisition; based onthe above detecting operations, determining said mismatch, and denyingsaid authentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was held facing towards a first direction during acquisition;detecting that a spatial-orientation sensor of the electronic device,indicates that the electronic device was held facing towards a seconddirection during acquisition; based on the above detecting operations,determining said mismatch, and denying said authentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was held steadily during acquisition; detecting that one or morenon-camera sensors of the electronic device, indicate that theelectronic device was held non-steadily during acquisition; based on theabove detecting operations, determining said mismatch, and denying saidauthentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was held non-steadily during acquisition; detecting that one ormore non-camera sensors of the electronic device, indicate that theelectronic device was held steadily during acquisition; based on theabove detecting operations, determining said mismatch, and denying saidauthentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was held during acquisition while being moved along a particularaxis of movement; detecting that one or more non-camera sensors of theelectronic device, indicate that the electronic device was held steadilyduring acquisition; based on the above detecting operations, determiningsaid mismatch, and denying said authentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was held during acquisition while being moved along a firstparticular axis of movement; detecting that one or more non-camerasensors of the electronic device, indicate that the electronic devicewas held during acquisition while being moved along a second particularaxis of movement that is different from said first particular axis ofmovement; based on the above detecting operations, determining saidmismatch, and denying said authentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was moved during acquisition in accordance with a first spatialmovement pattern; detecting that one or more non-camera sensors of theelectronic device, indicate that the electronic device was moved duringacquisition in accordance with a second, different, spatial movementpattern; based on the above detecting operations, determining saidmismatch, and denying said authentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was located indoors during acquisition; detecting that one ormore non-camera sensors of the electronic device, indicate that theelectronic device was located outdoors during acquisition; based on theabove detecting operations, determining said mismatch, and denying saidauthentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was located outdoors during acquisition; detecting that one ormore non-camera sensors of the electronic device, indicate that theelectronic device was located indoors during acquisition; based on theabove detecting operations, determining said mismatch, and denying saidauthentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was located at a first particular geographic location duringacquisition; detecting that one or more non-camera sensors of theelectronic device, indicate that the electronic device was located at asecond particular geographic location during acquisition; based on theabove detecting operations, determining said mismatch, and denying saidauthentication request.

In some embodiments, step (d) comprises: detecting that the content ofthe self-taken visual-acquisition input, indicates that the electronicdevice was moved during acquisition at a movement range of N degrees;detecting that one or more non-camera sensors of the electronic device,indicate that the electronic device was moved during acquisition at amovement range of M degrees, wherein the difference between M and N isat least P percent of N, wherein P and M and N are pre-defined values;based on the above detecting operations, determining said mismatch, anddenying said authentication request.

In some embodiments, step (d) comprises: determining that there exists acontradiction between (I) spatial orientation of the electronic deviceas sensed by one or more non-camera spatial orientation sensors of theelectronic device during acquisition, and (II) spatial orientation ofthe electronic device as determined from computer vision analysis ofcontent shown in said self-taken visual-acquisition input; based on saiddetermining, denying said authentication request.

In some embodiments, step (d) comprises: determining that there exists acontradiction between (I) spatial orientation of the electronic deviceas sensed by one or more gyroscope sensors of the electronic deviceduring acquisition, and (II) spatial orientation of the electronicdevice as determined from computer vision analysis of content shown insaid self-taken visual-acquisition input; based on said determining,denying said authentication request.

In some embodiments, step (d) comprises: determining that there exists acontradiction between (I) acceleration of the electronic device assensed by one or more accelerometers of the electronic device duringacquisition, and (II) acceleration of the electronic device asdetermined from computer vision analysis of content shown in saidself-taken visual-acquisition input; based on said determining, denyingsaid authentication request.

In some embodiments, step (d) comprises: determining that there exists acontradiction between (I) spatial spinning characteristic of theelectronic device as sensed by one or more spatial characteristicsensors of the electronic device during acquisition, and (II) spatialspinning characteristic of the electronic device as determined fromcomputer vision analysis of content shown in said self-takenvisual-acquisition input; based on said determining, denying saidauthentication request.

In some embodiments, step (d) comprises: determining that there exists acontradiction between (I) geographic location of the electronic deviceas sensed by a Global Positioning System (GPS) sensor of the electronicdevice during acquisition, and (II) geographic location of theelectronic device as determined from computer vision analysis of contentshown in said self-taken visual-acquisition input; based on saiddetermining, denying said authentication request.

In some embodiments, step (d) comprises: determining a contradictionbetween (A) output of an image recognition analysis applied to saidself-taken visual-acquisition input, and (B) data sensed by one or moreof (B1) accelerometer of the electronic device, (B2) gyroscope of theelectronic device, (B3) spatial-orientation sensor of said electronicdevice, (B4) Global Positioning System (GPS) sensor of the electronicdevice, (B5) location-determining sensor of the electronic device; basedon said contradiction, denying said authentication request.

In some embodiments, the user authentication process, that is based onsaid self-taken visual-acquisition input, is configured to authorize ordeny user access to at least one of: said electronic device (e.g., asmartphone, a tablet, a laptop computer, a desktop computer, asmart-watch, a gaming device), an electronic mail account, a socialnetwork account, an online retailer account, an online banking account,a vehicle, an ignition system or a starting system of a vehicle, aphysical venue or house or residence or office (e.g., protected orblocked by a door, a gate, a turnstile, or other obstacle or barrier).

Embodiments of the present invention may be utilized with a variety ofdevices or systems having a touch-screen or a touch-sensitive surface;for example, a smartphone, a cellular phone, a mobile phone, asmart-watch, a tablet, a handheld device, a portable electronic device,a portable gaming device, a portable audio/video player, an AugmentedReality (AR) device or headset or gear, a Virtual Reality (VR) device orheadset or gear, a “kiosk” type device, a vending machine, an AutomaticTeller Machine (ATM), a laptop computer, a desktop computer, a vehicularcomputer, a vehicular dashboard, a vehicular touch-screen, or the like.

Although portions of the discussion herein relate, for demonstrativepurposes, to wired links and/or wired communications, some embodimentsof the present invention are not limited in this regard, and may includeone or more wired or wireless links, may utilize one or more componentsof wireless communication, may utilize one or more methods or protocolsof wireless communication, or the like. Some embodiments may utilizewired communication and/or wireless communication.

The system(s) and/or device(s) of the present invention may optionallycomprise, or may be implemented by utilizing suitable hardwarecomponents and/or software components; for example, processors,processor cores, Central Processing Units (CPUs), Digital SignalProcessors (DSPs), circuits, Integrated Circuits (ICs), controllers,memory units, registers, accumulators, storage units, input units (e.g.,touch-screen, keyboard, keypad, stylus, mouse, touchpad, joystick,trackball, microphones), output units (e.g., screen, touch-screen,monitor, display unit, audio speakers), acoustic microphone(s) and/orsensor(s), optical microphone(s) and/or sensor(s), laser or laser-basedmicrophone(s) and/or sensor(s), wired or wireless modems or transceiversor transmitters or receivers, GPS receiver or GPS element or otherlocation-based or location-determining unit or system, network elements(e.g., routers, switches, hubs, antennas), and/or other suitablecomponents and/or modules.

The system(s) and/or devices of the present invention may optionally beimplemented by utilizing co-located components, remote components ormodules, “cloud computing” servers or devices or storage, client/serverarchitecture, peer-to-peer architecture, distributed architecture,and/or other suitable architectures or system topologies or networktopologies.

In accordance with embodiments of the present invention, calculations,operations and/or determinations may be performed locally within asingle device, or may be performed by or across multiple devices, or maybe performed partially locally and partially remotely (e.g., at a remoteserver) by optionally utilizing a communication channel to exchange rawdata and/or processed data and/or processing results.

Some embodiments may be implemented by using a special-purpose machineor a specific-purpose device that is not a generic computer, or by usinga non-generic computer or a non-general computer or machine. Such systemor device may utilize or may comprise one or more components or units ormodules that are not part of a “generic computer” and that are not partof a “general purpose computer”, for example, cellular transceivers,cellular transmitter, cellular receiver, GPS unit, location-determiningunit, accelerometer(s), gyroscope(s), device-orientation detectors orsensors, device-positioning detectors or sensors, or the like.

Some embodiments may be implemented as, or by utilizing, an automatedmethod or automated process, or a machine-implemented method or process,or as a semi-automated or partially-automated method or process, or as aset of steps or operations which may be executed or performed by acomputer or machine or system or other device.

Some embodiments may be implemented by using code or program code ormachine-readable instructions or machine-readable code, which may bestored on a non-transitory storage medium or non-transitory storagearticle (e.g., a CD-ROM, a DVD-ROM, a physical memory unit, a physicalstorage unit), such that the program or code or instructions, whenexecuted by a processor or a machine or a computer, cause such processoror machine or computer to perform a method or process as describedherein. Such code or instructions may be or may comprise, for example,one or more of: software, a software module, an application, a program,a subroutine, instructions, an instruction set, computing code, words,values, symbols, strings, variables, source code, compiled code,interpreted code, executable code, static code, dynamic code; including(but not limited to) code or instructions in high-level programminglanguage, low-level programming language, object-oriented programminglanguage, visual programming language, compiled programming language,interpreted programming language, C, C++, C#, Java, JavaScript, SQL,Ruby on Rails, Go, Cobol, Fortran, ActionScript, AJAX, XML, JSON, Lisp,Eiffel, Verilog, Hardware Description Language (HDL, BASIC, VisualBASIC, Matlab, Pascal, HTML, HTML5, CSS, Perl, Python, PHP, machinelanguage, machine code, assembly language, or the like.

Discussions herein utilizing terms such as, for example, “processing”,“computing”, “calculating”, “determining”, “establishing”, “analyzing”,“checking”, “detecting”, “measuring”, or the like, may refer tooperation(s) and/or process(es) of a processor, a computer, a computingplatform, a computing system, or other electronic device or computingdevice, that may automatically and/or autonomously manipulate and/ortransform data represented as physical (e.g., electronic) quantitieswithin registers and/or accumulators and/or memory units and/or storageunits into other data or that may perform other suitable operations.

Some embodiments of the present invention may perform steps oroperations such as, for example, “determining”, “identifying”,“comparing”, “checking”, “querying”, “searching”, “matching”, and/or“analyzing”, by utilizing, for example: a pre-defined threshold value towhich one or more parameter values may be compared; a comparison between(i) sensed or measured or calculated value(s), and (ii) pre-defined ordynamically-generated threshold value(s) and/or range values and/orupper limit value and/or lower limit value and/or maximum value and/orminimum value; a comparison or matching between sensed or measured orcalculated data, and one or more values as stored in a look-up table ora legend table or a list of reference value(s) or a database ofreference values or ranges; a comparison or matching or searchingprocess which searches for matches and/or identical results and/orsimilar results and/or sufficiently-close results, among multiple valuesor limits that are stored in a database or look-up table; utilization ofone or more equations, formula, weighted formula, and/or othercalculation in order to determine similarity or a match between or amongparameters or values; utilization of comparator units, lookup tables,threshold values, conditions, conditioning logic, Boolean operator(s)and/or other suitable components and/or operations.

The terms “plurality” and “a plurality”, as used herein, include, forexample, “multiple” or “two or more”. For example, “a plurality ofitems” includes two or more items.

References to “one embodiment”, “an embodiment”, “demonstrativeembodiment”, “various embodiments”, “some embodiments”, and/or similarterms, may indicate that the embodiment(s) so described may optionallyinclude a particular feature, structure, or characteristic, but notevery embodiment necessarily includes the particular feature, structure,or characteristic. Repeated use of the phrase “in one embodiment” doesnot necessarily refer to the same embodiment, although it may. Repeateduse of the phrase “in some embodiments” does not necessarily refer tothe same set or group of embodiments, although it may.

As used herein, and unless otherwise specified, the utilization ofordinal adjectives such as “first”, “second”, “third”, “fourth”, and soforth, to describe an item or an object, merely indicates that differentinstances of such like items or objects are being referred to; and doesnot intend to imply as if the items or objects so described must be in aparticular given sequence, either temporally, spatially, in ranking, orin any other ordering manner.

Some embodiments may comprise, or may be implemented by using, an “app”or application which may be downloaded or obtained from an “app store”or “applications store”, for free or for a fee, or which may bepre-installed on a computing device or electronic device, or which maybe transported to and/or installed on such computing device orelectronic device.

Functions, operations, components and/or features described herein withreference to one or more embodiments of the present invention, may becombined with, or may be utilized in combination with, one or more otherfunctions, operations, components and/or features described herein withreference to one or more other embodiments of the present invention.

While certain features of the present invention have been illustratedand described herein, many modifications, substitutions, changes, andequivalents may occur to those skilled in the art. Accordingly, theclaims are intended to cover all such modifications, substitutions,changes, and equivalents.

What is claimed is:
 1. A method comprising: (a) generating auser-authentication process for authenticating a user of an electronicdevice, wherein the user-authentication process utilizes a self-takenvisual-acquisition input of said user, wherein the self-takenvisual-acquisition input comprises one of: a self-taken image, aself-taken video; (b) performing image recognition on content shown insaid self-taken visual-acquisition input, and extracting a particularfeature based on image recognition of said content; (c) determining acharacteristic of said electronic device, based on data sensed by one ormore non-camera sensors of said electronic device during acquisition ofsaid self-taken visual-acquisition input; (d) detecting a mismatchbetween (I) said particular feature that was extracted in step (b) basedon image recognition of the content of the self-taken visual-acquisitioninput, and (II) said characteristic of the electronic device that wasdetermined in step (c) based on data sensed by said one or morenon-camera sensors of said electronic device; (e) based on said mismatchdetected in step (d), denying an authentication request received fromsaid electronic device wherein step (d) comprises: detecting that thecontent of the self-taken visual-acquisition input, indicates that theelectronic device was moved during acquisition at a movement range of Ndegrees; detecting that one or more non-camera sensors of the electronicdevice, indicate that the electronic device was moved during acquisitionat a movement range of M degrees, wherein the difference between M and Nis at least P percent of N, wherein P and M and N are pre-definedvalues; based on the above detecting operations, determining saidmismatch, and denying said authentication request.
 2. The method ofclaim 1, wherein step (d) comprises: detecting that the content of theself-taken visual-acquisition input, indicates that the electronicdevice was held vertically during acquisition; detecting that aspatial-orientation sensor of the electronic device, indicates that theelectronic device was held non-vertically during acquisition; based onthe above detecting operations, determining said mismatch, and denyingsaid authentication request.
 3. The method of claim 1, wherein step (d)comprises: detecting that the content of the self-takenvisual-acquisition input, indicates that the electronic device was heldhorizontally during acquisition; detecting that a spatial-orientationsensor of the electronic device, indicates that the electronic devicewas held non-horizontally during acquisition; based on the abovedetecting operations, determining said mismatch, and denying saidauthentication request.
 4. The method of claim 1, wherein step (d)comprises: determining that there exists a contradiction between (I)spatial orientation of the electronic device as sensed by one or morenon-camera spatial orientation sensors of the electronic device duringacquisition, and (II) spatial orientation of the electronic device asdetermined from computer vision analysis of content shown in saidself-taken visual-acquisition input; based on said determining, denyingsaid authentication request.
 5. The method of claim 1, wherein step (d)comprises: determining that there exists a contradiction between (I)spatial orientation of the electronic device as sensed by one or moregyroscope sensors of the electronic device during acquisition, and (II)spatial orientation of the electronic device as determined from computervision analysis of content shown in said self-taken visual-acquisitioninput; based on said determining, denying said authentication request.6. The method of claim 1, wherein step (d) comprises: determining acontradiction between (A) output of an image recognition analysisapplied to said self-taken visual-acquisition input, and (B) data sensedby one or more of (B1) accelerometer of the electronic device, (B2)gyroscope of the electronic device, (B3) spatial-orientation sensor ofsaid electronic device, (B4) Global Positioning System (GPS) sensor ofthe electronic device, (B5) location-determining sensor of theelectronic device; based on said contradiction, denying saidauthentication request.
 7. The method of claim 1, wherein the userauthentication process, that is based on said self-takenvisual-acquisition input, is configured to authorize or deny user accessto at least one of: said electronic device, an electronic mail account,a social network account, an online retailer account, an online bankingaccount, a vehicle, a physical venue.
 8. A non-transitory storage mediumhaving stored thereon instructions that, when executed by one or morehardware processors, cause said one or more hardware processors toperform a method comprising: (a) generating a user-authenticationprocess for authenticating a user of an electronic device, wherein theuser-authentication process utilizes a self-taken visual-acquisitioninput of said user, wherein the self-taken visual-acquisition inputcomprises one of: a self-taken image, a self-taken video; (b) performingimage recognition on content shown in said self-taken visual-acquisitioninput, and extracting a particular feature based on image recognition ofsaid content; (c) determining a characteristic of said electronicdevice, based on data sensed by one or more non-camera sensors of saidelectronic device during acquisition of said self-takenvisual-acquisition input; (d) detecting a mismatch between (I) saidparticular feature that was extracted in step (b) based on imagerecognition of the content of the self-taken visual-acquisition input,and (II) said characteristic of the electronic device that wasdetermined in step (c) based on data sensed by said one or morenon-camera sensors of said electronic device; (e) based on said mismatchdetected in step (d), denying an authentication request received fromsaid electronic device; wherein step (d) comprises: detecting that thecontent of the self-taken visual-acquisition input, indicates that theelectronic device was moved during acquisition at a movement range of Ndegrees; detecting that one or more non-camera sensors of the electronicdevice, indicate that the electronic device was moved during acquisitionat a movement range of M degrees, wherein the difference between M and Nis at least P percent of N, wherein P and M and N are pre-definedvalues; based on the above detecting operations, determining saidmismatch, and denying said authentication request.
 9. A systemcomprising: one or more processors, operably associated with one or morememory units, wherein the one or more processors are configured: (a) togenerate a user-authentication process for authenticating a user of anelectronic device, wherein the user-authentication process utilizes aself-taken visual-acquisition input of said user, wherein the self-takenvisual-acquisition input comprises one of: a self-taken image, aself-taken video; (b) to perform image recognition on content shown insaid self-taken visual-acquisition input, and to extract a particularfeature based on image recognition of said content; (c) to determine acharacteristic of said electronic device, based on data sensed by one ormore non-camera sensors of said electronic device during acquisition ofsaid self-taken visual-acquisition input; (d) to detect a mismatchbetween (I) said particular feature that was extracted in step (b) basedon image recognition of the content of the self-taken visual-acquisitioninput, and (II) said characteristic of the electronic device that wasdetermined in step (c) based on data sensed by said one or morenon-camera sensors of said electronic device; (e) based on saidmismatch, to deny an authentication request received from saidelectronic device; wherein in (d) the one or more processors areconfigured: to detect that the content of the self-takenvisual-acquisition input, indicates that the electronic device was movedduring acquisition at a movement range of N degrees; to detect that oneor more non-camera sensors of the electronic device, indicate that theelectronic device was moved during acquisition at a movement range of Mdegrees, wherein the difference between M and N is at least P percent ofN, wherein P and M and N are pre-defined values; based on the above, todetermine said mismatch, and to deny said authentication request.